2023-11-02T15:13:32.106 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-02T15:13:32.108 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599 --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/orig.config.yaml 2023-11-02T15:13:32.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-02T15:13:32.253 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599 branch: wip-yuri6-testing-2023-11-01-0745-reef description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} email: null first_in_suite: false job_id: '7444599' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2023-11-01-0745-reef ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 120 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2023-11-01-0745-reef sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 owner: scheduled_yuriw@teuthology priority: 49 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 525 sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2023-11-01-0745-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 targets: smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBugmfG+rwui8Ts34mPlTCz0WfNTdtwY57m8tJjMFcAuonN4c0UAtSWyRtuAfmNqAPq1XJ7wLMHJ0GnsykGZZe8= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgiUDqKH+Rx2Vp7SONDWsX0Ye5bRtCPgwzqtDCC/9kP7WEpPClEKou7KBljVcDfFamD9gd/+RNBQ/+LSXdMf10= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-02_14:20:05 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-11-02T15:13:32.254 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa; will attempt to use it 2023-11-02T15:13:32.255 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks 2023-11-02T15:13:32.255 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-02T15:13:32.257 INFO:teuthology.task.internal:Checking packages... 2023-11-02T15:13:32.283 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081' 2023-11-02T15:13:32.283 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-02T15:13:32.284 INFO:teuthology.packaging:ref: None 2023-11-02T15:13:32.285 INFO:teuthology.packaging:tag: None 2023-11-02T15:13:32.285 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-11-01-0745-reef 2023-11-02T15:13:32.285 INFO:teuthology.packaging:sha1: b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:13:32.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2023-11-01-0745-reef 2023-11-02T15:13:32.436 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1159.gb5a9486b 2023-11-02T15:13:32.436 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-02T15:13:32.448 INFO:teuthology.task.internal:no buildpackages task found 2023-11-02T15:13:32.448 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-02T15:13:32.470 INFO:teuthology.task.internal:Saving configuration 2023-11-02T15:13:32.483 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-02T15:13:32.491 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-02T15:13:32.516 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-02 15:08:37.119118', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBugmfG+rwui8Ts34mPlTCz0WfNTdtwY57m8tJjMFcAuonN4c0UAtSWyRtuAfmNqAPq1XJ7wLMHJ0GnsykGZZe8='} 2023-11-02T15:13:32.536 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-02 15:08:37.120312', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgiUDqKH+Rx2Vp7SONDWsX0Ye5bRtCPgwzqtDCC/9kP7WEpPClEKou7KBljVcDfFamD9gd/+RNBQ/+LSXdMf10='} 2023-11-02T15:13:32.536 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-02T15:13:32.546 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-11-02T15:13:32.547 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-11-02T15:13:32.547 INFO:teuthology.run_tasks:Running task console_log... 2023-11-02T15:13:32.624 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f22ac17ea60>, signals=[15]) 2023-11-02T15:13:32.625 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-02T15:13:32.630 INFO:teuthology.task.internal:Opening connections... 2023-11-02T15:13:32.631 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:13:32.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:13:32.717 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:13:32.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:13:32.805 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-02T15:13:32.811 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2023-11-02T15:13:32.829 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2023-11-02T15:13:32.829 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-11-02T15:13:32.885 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2023-11-02T15:13:32.885 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2023-11-02T15:13:32.885 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2023-11-02T15:13:32.886 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2023-11-02T15:13:32.886 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2023-11-02T15:13:32.886 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2023-11-02T15:13:32.886 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-02T15:13:32.886 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2023-11-02T15:13:32.887 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2023-11-02T15:13:32.887 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-02T15:13:32.887 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2023-11-02T15:13:32.887 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-02T15:13:32.887 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-02T15:13:32.888 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-02T15:13:32.888 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-11-02T15:13:32.914 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2023-11-02T15:13:32.932 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2023-11-02T15:13:32.932 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2023-11-02T15:13:32.989 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2023-11-02T15:13:32.989 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2023-11-02T15:13:32.989 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2023-11-02T15:13:32.989 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2023-11-02T15:13:32.989 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2023-11-02T15:13:32.990 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2023-11-02T15:13:32.990 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-02T15:13:32.990 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2023-11-02T15:13:32.990 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2023-11-02T15:13:32.991 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-02T15:13:32.991 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2023-11-02T15:13:32.991 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-02T15:13:32.991 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-02T15:13:32.991 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-02T15:13:32.992 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-11-02T15:13:33.014 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-02T15:13:33.023 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-02T15:13:33.029 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-02T15:13:33.029 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2023-11-02T15:13:33.032 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2023-11-02T15:13:33.048 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-02T15:13:33.054 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-02T15:13:33.054 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2023-11-02T15:13:33.088 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2023-11-02T15:13:33.208 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-02T15:13:33.266 INFO:teuthology.run_tasks:Running task kernel... 2023-11-02T15:13:33.282 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-02T15:13:33.283 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-02T15:13:33.283 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-02T15:13:33.284 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-02T15:13:33.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T15:13:33.301 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2023-11-02T15:13:33.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T15:13:33.302 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2023-11-02T15:13:33.360 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-361.el9.x86_64 2023-11-02T15:13:33.360 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-361.el9.x86_64 2023-11-02T15:13:33.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2023-11-02T15:13:33.362 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-361.el9.x86_64 2023-11-02T15:13:33.362 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-361.el9.x86_64 2023-11-02T15:13:33.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2023-11-02T15:13:35.427 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-11-02T15:13:35.453 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-11-02T15:13:36.744 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - BaseOS 8.0 MB/s | 2.0 MB 00:00 2023-11-02T15:13:36.760 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2023-11-02T15:13:37.777 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 7.9 MB 00:00 2023-11-02T15:13:37.809 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 7.9 MB 00:00 2023-11-02T15:13:40.227 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 13 MB/s | 18 MB 00:01 2023-11-02T15:13:42.685 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 18 MB 00:03 2023-11-02T15:13:47.061 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 5.8 MB 00:03 2023-11-02T15:13:48.452 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 5.8 MB 00:02 2023-11-02T15:13:48.519 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2023-11-02T15:13:49.796 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 17 MB/s | 19 MB 00:01 2023-11-02T15:13:49.961 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2023-11-02T15:13:51.215 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 17 MB/s | 19 MB 00:01 2023-11-02T15:13:54.338 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2023-11-02T15:13:55.770 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-11-02T15:13:56.327 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-02T15:13:56.327 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-02T15:13:56.428 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:13:56.433 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:13:56.434 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2023-11-02T15:13:56.434 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:13:56.434 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:13:56.434 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-378.el9 baseos 5.3 M 2023-11-02T15:13:56.434 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:13:56.435 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core x86_64 5.14.0-378.el9 baseos 20 M 2023-11-02T15:13:56.435 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules x86_64 5.14.0-378.el9 baseos 38 M 2023-11-02T15:13:56.435 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core x86_64 5.14.0-378.el9 baseos 32 M 2023-11-02T15:13:56.435 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:13:56.435 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:13:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:13:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages 2023-11-02T15:13:56.436 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:13:56.437 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 95 M 2023-11-02T15:13:56.437 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 120 M 2023-11-02T15:13:56.437 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:13:57.430 INFO:teuthology.orchestra.run.smithi174.stdout:(1/4): kernel-5.14.0-378.el9.x86_64.rpm 6.7 MB/s | 5.3 MB 00:00 2023-11-02T15:13:57.785 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-02T15:13:57.786 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-02T15:13:57.879 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:13:57.886 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:13:57.886 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-11-02T15:13:57.887 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:13:57.887 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:13:57.887 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-378.el9 baseos 5.3 M 2023-11-02T15:13:57.887 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:13:57.887 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-378.el9 baseos 20 M 2023-11-02T15:13:57.888 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-378.el9 baseos 38 M 2023-11-02T15:13:57.888 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-378.el9 baseos 32 M 2023-11-02T15:13:57.888 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:13:57.888 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:13:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:13:57.889 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2023-11-02T15:13:57.889 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:13:57.890 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 95 M 2023-11-02T15:13:57.890 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 120 M 2023-11-02T15:13:57.890 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:13:58.172 INFO:teuthology.orchestra.run.smithi174.stdout:(2/4): kernel-core-5.14.0-378.el9.x86_64.rpm 13 MB/s | 20 MB 00:01 2023-11-02T15:13:58.755 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-378.el9.x86_64.rpm 6.5 MB/s | 5.3 MB 00:00 2023-11-02T15:13:58.814 INFO:teuthology.orchestra.run.smithi174.stdout:(3/4): kernel-modules-5.14.0-378.el9.x86_64.rpm 17 MB/s | 38 MB 00:02 2023-11-02T15:13:59.239 INFO:teuthology.orchestra.run.smithi174.stdout:(4/4): kernel-modules-core-5.14.0-378.el9.x86_6 18 MB/s | 32 MB 00:01 2023-11-02T15:13:59.240 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:13:59.240 INFO:teuthology.orchestra.run.smithi174.stdout:Total 34 MB/s | 95 MB 00:02 2023-11-02T15:14:00.088 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:14:00.105 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-378.el9.x86_64.rpm 9.2 MB/s | 20 MB 00:02 2023-11-02T15:14:00.450 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:14:00.451 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:14:01.330 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-5.14.0-378.el9.x86_64.rpm 11 MB/s | 38 MB 00:03 2023-11-02T15:14:01.605 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-core-5.14.0-378.el9.x86_6 11 MB/s | 32 MB 00:02 2023-11-02T15:14:01.606 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:14:01.606 INFO:teuthology.orchestra.run.smithi175.stdout:Total 26 MB/s | 95 MB 00:03 2023-11-02T15:14:01.755 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:14:01.756 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:14:02.452 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:14:02.797 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:14:02.797 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:14:04.113 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:14:04.114 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:14:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:14:05.309 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-core-5.14.0-378.el9.x86_64 1/4 2023-11-02T15:14:05.447 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:06.749 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:14:07.532 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-378.el9.x86_64 1/4 2023-11-02T15:14:07.696 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:08.400 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:10.550 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:10.721 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:13.021 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:13.138 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:15.292 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:17.743 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:20.046 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:46.516 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:46.557 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:46.919 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:46.919 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-378.el9.x86_64 1/4 2023-11-02T15:14:46.920 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:46.920 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:14:47.337 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:14:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core-5.14.0-378.el9.x86_64 2023-11-02T15:14:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-5.14.0-378.el9.x86_64 2023-11-02T15:14:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core-5.14.0-378.el9.x86_64 2023-11-02T15:14:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:14:47.338 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:14:47.806 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2023-11-02T15:14:47.947 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:47.948 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-11-02T15:14:47.948 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-378.el9.x86_64 2023-11-02T15:14:47.948 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-02T15:14:47.949 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-02T15:14:47.949 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-02T15:14:47.949 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2023-11-02T15:14:47.949 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2023-11-02T15:14:48.065 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:48.066 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-11-02T15:14:48.066 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2023-11-02T15:14:48.105 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2023-11-02T15:14:48.106 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2023-11-02T15:14:48.106 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2023-11-02T15:14:48.106 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 68.el9 2023-11-02T15:14:48.106 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2023-11-02T15:14:48.107 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-02T15:14:48.107 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2023-11-02T15:14:48.107 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8264254 2023-11-02T15:14:48.107 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2023-11-02T15:14:48.107 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-02T15:14:48.108 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-02T15:14:48.108 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-02T15:14:48.108 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-02T15:14:48.108 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2023-11-02T15:14:48.109 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2023-11-02T15:14:48.109 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-02T15:14:48.109 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2023-11-02T15:14:48.109 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2023-11-02T15:14:48.110 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:14:48.110 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-02T15:14:48.110 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-02T15:14:48.110 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-02T15:14:48.110 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2023-11-02T15:14:48.111 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:14:48.111 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2023-11-02T15:14:48.112 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-02T15:14:48.112 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 5.14.0-378.el9.x86_64 2023-11-02T15:14:48.112 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-02T15:14:48.756 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2023-11-02T15:14:49.200 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:49.242 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-378.el9.x86_64 1/4 2023-11-02T15:14:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-11-02T15:14:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-11-02T15:14:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-11-02T15:14:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:14:50.077 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:14:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:14:50.079 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:14:50.135 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-02T15:14:50.162 INFO:teuthology.orchestra.run.smithi174.stderr:done 2023-11-02T15:14:50.165 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2023-11-02T15:14:50.181 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.50qQIHFqEU 2023-11-02T15:14:50.181 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.50qQIHFqEU 2023-11-02T15:14:50.249 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.50qQIHFqEU 2023-11-02T15:14:50.380 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.50qQIHFqEU is 6KB 2023-11-02T15:14:50.400 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.50qQIHFqEU 2023-11-02T15:14:50.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2023-11-02T15:14:50.479 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-02T15:14:50.479 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-02T15:14:50.480 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-02T15:14:50.480 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64.conf 2023-11-02T15:14:50.481 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.564 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2023-11-02T15:14:50.637 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2023-11-02T15:14:50.705 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.706 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-11-02T15:14:50.706 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-378.el9.x86_64 2023-11-02T15:14:50.707 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-02T15:14:50.707 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-02T15:14:50.708 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-02T15:14:50.708 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2023-11-02T15:14:50.708 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2023-11-02T15:14:50.826 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-378.el9.x86_64 2023-11-02T15:14:50.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-11-02T15:14:50.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2023-11-02T15:14:50.861 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2023-11-02T15:14:50.862 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2023-11-02T15:14:50.862 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2023-11-02T15:14:50.862 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2023-11-02T15:14:50.862 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2023-11-02T15:14:50.863 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-02T15:14:50.863 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2023-11-02T15:14:50.863 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2023-11-02T15:14:50.863 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2023-11-02T15:14:50.864 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-02T15:14:50.864 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-02T15:14:50.864 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-02T15:14:50.864 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-02T15:14:50.864 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2023-11-02T15:14:50.865 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2023-11-02T15:14:50.865 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-02T15:14:50.865 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2023-11-02T15:14:50.865 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2023-11-02T15:14:50.865 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:14:50.866 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2023-11-02T15:14:50.867 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-02T15:14:50.868 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-378.el9.x86_64 2023-11-02T15:14:50.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-02T15:14:51.474 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2023-11-02T15:14:52.985 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-02T15:14:53.015 INFO:teuthology.orchestra.run.smithi175.stderr:done 2023-11-02T15:14:53.017 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2023-11-02T15:14:53.073 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.UchNyvSIRL 2023-11-02T15:14:53.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UchNyvSIRL 2023-11-02T15:14:53.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.UchNyvSIRL 2023-11-02T15:14:53.279 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.UchNyvSIRL is 6KB 2023-11-02T15:14:53.328 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.UchNyvSIRL 2023-11-02T15:14:53.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2023-11-02T15:14:53.410 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-02T15:14:53.410 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-02T15:14:53.410 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-02T15:14:53.411 INFO:teuthology.orchestra.run.smithi175.stdout:15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64.conf 2023-11-02T15:14:53.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64 2023-11-02T15:14:53.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2023-11-02T15:15:20.666 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-02T15:15:20.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:15:20.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:15:23.566 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-02T15:15:23.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:15:23.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:15:55.135 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-11-02T15:15:58.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-11-02T15:16:04.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:16:04.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:16:07.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:16:07.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:16:22.522 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-11-02T15:16:25.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2023-11-02T15:16:34.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:16:34.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:16:35.010 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-11-02T15:16:35.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:16:35.404 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-378.el9.x86_64"... 2023-11-02T15:16:35.404 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2023-11-02T15:16:35.418 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-378.el9.x86_64 2023-11-02T15:16:35.419 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-378.el9.x86_64 vs 5.14.0-378.el9.x86_64 2023-11-02T15:16:35.419 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-02T15:16:35.420 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-02T15:16:36.421 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-02T15:16:36.421 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-02T15:16:36.524 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2023-11-02T15:16:36.566 DEBUG:teuthology.parallel:result is None 2023-11-02T15:16:37.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:16:37.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:16:38.060 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-11-02T15:16:38.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:16:38.444 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-378.el9.x86_64"... 2023-11-02T15:16:38.445 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2023-11-02T15:16:38.459 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-378.el9.x86_64 2023-11-02T15:16:38.459 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-378.el9.x86_64 vs 5.14.0-378.el9.x86_64 2023-11-02T15:16:38.459 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-02T15:16:38.460 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-02T15:16:39.461 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-02T15:16:39.461 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-02T15:16:39.527 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2023-11-02T15:16:39.568 DEBUG:teuthology.parallel:result is None 2023-11-02T15:16:39.569 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-02T15:16:39.576 INFO:teuthology.task.internal:Creating test directory... 2023-11-02T15:16:39.576 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-02T15:16:39.579 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-02T15:16:39.594 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-02T15:16:39.605 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-02T15:16:39.619 INFO:teuthology.task.internal:Creating archive directory... 2023-11-02T15:16:39.620 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-02T15:16:39.632 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-02T15:16:39.658 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-02T15:16:39.669 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-02T15:16:39.670 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-02T15:16:39.684 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-02T15:16:39.713 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-02T15:16:39.724 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-02T15:16:39.726 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-02T15:16:39.737 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-02T15:16:39.738 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-02T15:16:39.750 INFO:teuthology.task.internal:Configuring sudo... 2023-11-02T15:16:39.750 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-02T15:16:39.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-02T15:16:39.810 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-02T15:16:39.823 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-02T15:16:39.824 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-02T15:16:39.829 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-02T15:16:39.869 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-02T15:16:39.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-02T15:16:39.966 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-02T15:16:40.044 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-02T15:16:40.082 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:16:40.083 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-02T15:16:40.150 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-02T15:16:40.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-02T15:16:40.247 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-02T15:16:40.333 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-02T15:16:40.380 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:16:40.381 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-02T15:16:40.451 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2023-11-02T15:16:40.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2023-11-02T15:16:40.498 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-02T15:16:40.542 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-02T15:16:41.036 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-02T15:16:41.046 INFO:teuthology.task.internal:Starting timer... 2023-11-02T15:16:41.046 INFO:teuthology.run_tasks:Running task pcp... 2023-11-02T15:16:41.065 INFO:teuthology.run_tasks:Running task selinux... 2023-11-02T15:16:41.077 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2023-11-02T15:16:41.139 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2023-11-02T15:16:41.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2023-11-02T15:16:41.255 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2023-11-02T15:16:41.256 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-02T15:16:41.256 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2023-11-02T15:16:41.283 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2023-11-02T15:16:41.283 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2023-11-02T15:16:41.323 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2023-11-02T15:16:41.324 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi174.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2023-11-02T15:16:41.324 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-02T15:16:41.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T15:16:41.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-02T15:16:41.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T15:16:41.394 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-02T15:16:41.394 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2023-11-02T15:16:41.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2023-11-02T15:16:41.675 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-02T15:16:41.686 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-02T15:16:41.686 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-02T15:16:41.706 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-02T15:16:41.707 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi174.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-02T15:22:08.004 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi174.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2023-11-02T15:22:08.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:22:08.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:22:08.088 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-11-02T15:22:08.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-02T15:22:08.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:22:08.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-02T15:22:08.237 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-11-02T15:22:08.305 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-11-02T15:22:08.305 INFO:teuthology.run_tasks:Running task clock... 2023-11-02T15:22:08.321 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-02T15:22:08.321 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-02T15:22:08.322 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-02T15:22:08.324 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-02T15:22:08.325 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-02T15:22:08.353 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-02T15:22:08.369 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-02T15:22:08.375 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-02T15:22:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-02T15:22:08.397 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2023-11-02T15:22:08.411 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2023-11-02T15:22:08.420 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2023-11-02T15:22:08.427 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-02T15:22:08.433 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2023-11-02T15:22:08.443 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-02T15:22:08.449 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-02T15:22:08.466 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-02T15:22:08.486 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2023-11-02T15:22:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-02T15:22:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-02T15:22:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.508 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2023-11-02T15:22:08.512 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-02T15:22:08.512 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-11-02T15:22:08.512 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.512 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.513 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.513 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T15:22:08.513 INFO:teuthology.run_tasks:Running task install... 2023-11-02T15:22:08.521 DEBUG:teuthology.task.install:project ceph 2023-11-02T15:22:08.522 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081'}} 2023-11-02T15:22:08.522 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081'} 2023-11-02T15:22:08.523 INFO:teuthology.task.install:Using flavor: default 2023-11-02T15:22:08.528 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2023-11-02T15:22:08.528 INFO:teuthology.task.install:extra packages: [] 2023-11-02T15:22:08.529 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081', 'tag': None, 'wait_for_package': False} 2023-11-02T15:22:08.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:22:08.532 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081', 'tag': None, 'wait_for_package': False} 2023-11-02T15:22:08.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:22:08.768 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/ 2023-11-02T15:22:08.769 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1159.gb5a9486b 2023-11-02T15:22:08.782 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/ 2023-11-02T15:22:08.783 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1159.gb5a9486b 2023-11-02T15:22:08.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-02T15:22:08.897 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:22:08.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-02T15:22:08.924 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-11-02T15:22:08.925 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-02T15:22:08.950 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-11-01-0745-reef/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-02T15:22:08.950 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:22:08.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-02T15:22:08.978 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-11-02T15:22:08.978 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b5a9486b4a6f144f83c51b7ec33111a2c86dc081/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-02T15:22:08.992 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-02T15:22:09.046 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-02T15:22:09.071 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-02T15:22:09.126 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-02T15:22:09.137 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2023-11-02T15:22:09.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2023-11-02T15:22:09.192 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2023-11-02T15:22:09.194 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2023-11-02T15:22:09.380 INFO:teuthology.orchestra.run.smithi175.stdout:72 files removed 2023-11-02T15:22:09.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw 2023-11-02T15:22:09.431 INFO:teuthology.orchestra.run.smithi174.stdout:72 files removed 2023-11-02T15:22:09.457 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw 2023-11-02T15:22:10.270 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 156 kB/s | 77 kB 00:00 2023-11-02T15:22:10.273 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 142 kB/s | 77 kB 00:00 2023-11-02T15:22:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-11-02T15:22:10.707 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-11-02T15:22:11.054 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2023-11-02T15:22:11.092 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2023-11-02T15:22:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-11-02T15:22:11.556 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-11-02T15:22:12.940 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2023-11-02T15:22:12.944 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2023-11-02T15:22:14.198 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 7.9 MB 00:00 2023-11-02T15:22:14.629 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 7.9 MB 00:01 2023-11-02T15:22:16.415 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 16 MB/s | 18 MB 00:01 2023-11-02T15:22:16.536 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-11-02T15:22:21.217 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.8 MB 00:01 2023-11-02T15:22:21.853 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 5.8 MB 00:02 2023-11-02T15:22:22.575 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2023-11-02T15:22:23.171 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 15 kB 00:00 2023-11-02T15:22:24.196 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 13 MB/s | 19 MB 00:01 2023-11-02T15:22:24.881 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 12 MB/s | 19 MB 00:01 2023-11-02T15:22:28.694 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2023-11-02T15:22:29.398 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2023-11-02T15:22:31.077 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:22:31.081 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:31.081 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:22:31.081 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:31.081 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:22:31.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 7.7 M 2023-11-02T15:22:31.082 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2023-11-02T15:22:31.082 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.3 M 2023-11-02T15:22:31.082 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.0 M 2023-11-02T15:22:31.082 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:22:31.083 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options 2023-11-02T15:22:31.083 INFO:teuthology.orchestra.run.smithi174.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-02T15:22:31.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 5.2 M 2023-11-02T15:22:31.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 18 M 2023-11-02T15:22:31.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 24 k 2023-11-02T15:22:31.084 INFO:teuthology.orchestra.run.smithi174.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-11-02T15:22:31.084 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-02T15:22:31.084 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-02T15:22:31.084 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-02T15:22:31.084 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 659 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 475 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-02T15:22:31.085 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.4 M 2023-11-02T15:22:31.086 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-02T15:22:31.086 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-02T15:22:31.086 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-02T15:22:31.086 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse 2023-11-02T15:22:31.086 INFO:teuthology.orchestra.run.smithi174.stdout: x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 45 k 2023-11-02T15:22:31.087 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 124 k 2023-11-02T15:22:31.087 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 162 k 2023-11-02T15:22:31.087 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-02T15:22:31.087 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 320 k 2023-11-02T15:22:31.087 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 298 k 2023-11-02T15:22:31.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 99 k 2023-11-02T15:22:31.088 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-02T15:22:31.088 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-02T15:22:31.088 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:22:31.089 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:31.089 INFO:teuthology.orchestra.run.smithi174.stdout:Install 28 Packages 2023-11-02T15:22:31.089 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 2 Packages 2023-11-02T15:22:31.089 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:31.090 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 53 M 2023-11-02T15:22:31.090 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:22:31.766 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:22:31.769 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:31.770 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:22:31.770 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:31.770 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:22:31.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 7.7 M 2023-11-02T15:22:31.770 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2023-11-02T15:22:31.771 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.3 M 2023-11-02T15:22:31.771 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.0 M 2023-11-02T15:22:31.771 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:22:31.771 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options 2023-11-02T15:22:31.773 INFO:teuthology.orchestra.run.smithi175.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-02T15:22:31.773 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 5.2 M 2023-11-02T15:22:31.773 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 18 M 2023-11-02T15:22:31.773 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 24 k 2023-11-02T15:22:31.774 INFO:teuthology.orchestra.run.smithi175.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-11-02T15:22:31.774 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-02T15:22:31.774 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-02T15:22:31.774 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-02T15:22:31.774 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 659 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 475 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-02T15:22:31.775 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.4 M 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 45 k 2023-11-02T15:22:31.776 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 124 k 2023-11-02T15:22:31.777 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 162 k 2023-11-02T15:22:31.777 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-02T15:22:31.777 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 320 k 2023-11-02T15:22:31.777 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 298 k 2023-11-02T15:22:31.777 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 99 k 2023-11-02T15:22:31.778 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-02T15:22:31.778 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-02T15:22:31.778 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:31.778 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:22:31.778 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:31.779 INFO:teuthology.orchestra.run.smithi175.stdout:Install 28 Packages 2023-11-02T15:22:31.779 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2023-11-02T15:22:31.779 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:31.780 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 53 M 2023-11-02T15:22:31.780 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:22:31.962 INFO:teuthology.orchestra.run.smithi174.stdout:(1/30): ceph-radosgw-18.2.0-1159.gb5a9486b.el9. 12 MB/s | 7.7 MB 00:00 2023-11-02T15:22:32.003 INFO:teuthology.orchestra.run.smithi174.stdout:(2/30): ceph-selinux-18.2.0-1159.gb5a9486b.el9. 579 kB/s | 24 kB 00:00 2023-11-02T15:22:32.079 INFO:teuthology.orchestra.run.smithi174.stdout:(3/30): libcephfs2-18.2.0-1159.gb5a9486b.el9.x8 8.6 MB/s | 659 kB 00:00 2023-11-02T15:22:32.137 INFO:teuthology.orchestra.run.smithi174.stdout:(4/30): libradosstriper1-18.2.0-1159.gb5a9486b. 8.0 MB/s | 475 kB 00:00 2023-11-02T15:22:32.437 INFO:teuthology.orchestra.run.smithi174.stdout:(5/30): librgw2-18.2.0-1159.gb5a9486b.el9.x86_6 15 MB/s | 4.4 MB 00:00 2023-11-02T15:22:32.479 INFO:teuthology.orchestra.run.smithi174.stdout:(6/30): python3-ceph-argparse-18.2.0-1159.gb5a9 1.1 MB/s | 45 kB 00:00 2023-11-02T15:22:32.530 INFO:teuthology.orchestra.run.smithi174.stdout:(7/30): python3-ceph-common-18.2.0-1159.gb5a948 2.4 MB/s | 124 kB 00:00 2023-11-02T15:22:32.588 INFO:teuthology.orchestra.run.smithi174.stdout:(8/30): python3-cephfs-18.2.0-1159.gb5a9486b.el 2.7 MB/s | 162 kB 00:00 2023-11-02T15:22:32.655 INFO:teuthology.orchestra.run.smithi174.stdout:(9/30): python3-rados-18.2.0-1159.gb5a9486b.el9 4.7 MB/s | 320 kB 00:00 2023-11-02T15:22:32.722 INFO:teuthology.orchestra.run.smithi174.stdout:(10/30): python3-rbd-18.2.0-1159.gb5a9486b.el9. 4.4 MB/s | 298 kB 00:00 2023-11-02T15:22:32.781 INFO:teuthology.orchestra.run.smithi174.stdout:(11/30): python3-rgw-18.2.0-1159.gb5a9486b.el9. 1.7 MB/s | 99 kB 00:00 2023-11-02T15:22:32.873 INFO:teuthology.orchestra.run.smithi174.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 1.1 MB/s | 106 kB 00:00 2023-11-02T15:22:32.914 INFO:teuthology.orchestra.run.smithi174.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2023-11-02T15:22:32.973 INFO:teuthology.orchestra.run.smithi174.stdout:(14/30): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-11-02T15:22:33.015 INFO:teuthology.orchestra.run.smithi174.stdout:(15/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2023-11-02T15:22:33.049 INFO:teuthology.orchestra.run.smithi174.stdout:(16/30): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 41 kB 00:00 2023-11-02T15:22:33.224 INFO:teuthology.orchestra.run.smithi174.stdout:(17/30): ceph-base-18.2.0-1159.gb5a9486b.el9.x8 2.7 MB/s | 5.2 MB 00:01 2023-11-02T15:22:33.257 INFO:teuthology.orchestra.run.smithi174.stdout:(18/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2023-11-02T15:22:33.515 INFO:teuthology.orchestra.run.smithi175.stdout:(1/30): ceph-common-18.2.0-1159.gb5a9486b.el9.x 12 MB/s | 18 MB 00:01 2023-11-02T15:22:33.565 INFO:teuthology.orchestra.run.smithi175.stdout:(2/30): ceph-selinux-18.2.0-1159.gb5a9486b.el9. 484 kB/s | 24 kB 00:00 2023-11-02T15:22:33.632 INFO:teuthology.orchestra.run.smithi175.stdout:(3/30): libcephfs2-18.2.0-1159.gb5a9486b.el9.x8 9.7 MB/s | 659 kB 00:00 2023-11-02T15:22:33.699 INFO:teuthology.orchestra.run.smithi175.stdout:(4/30): libradosstriper1-18.2.0-1159.gb5a9486b. 7.0 MB/s | 475 kB 00:00 2023-11-02T15:22:33.750 INFO:teuthology.orchestra.run.smithi174.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 211 kB/s | 111 kB 00:00 2023-11-02T15:22:33.908 INFO:teuthology.orchestra.run.smithi174.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 475 kB/s | 309 kB 00:00 2023-11-02T15:22:33.975 INFO:teuthology.orchestra.run.smithi175.stdout:(5/30): librgw2-18.2.0-1159.gb5a9486b.el9.x86_6 16 MB/s | 4.4 MB 00:00 2023-11-02T15:22:34.025 INFO:teuthology.orchestra.run.smithi175.stdout:(6/30): python3-ceph-argparse-18.2.0-1159.gb5a9 901 kB/s | 45 kB 00:00 2023-11-02T15:22:34.050 INFO:teuthology.orchestra.run.smithi174.stdout:(21/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 172 kB/s | 24 kB 00:00 2023-11-02T15:22:34.067 INFO:teuthology.orchestra.run.smithi175.stdout:(7/30): python3-ceph-common-18.2.0-1159.gb5a948 2.9 MB/s | 124 kB 00:00 2023-11-02T15:22:34.109 INFO:teuthology.orchestra.run.smithi175.stdout:(8/30): python3-cephfs-18.2.0-1159.gb5a9486b.el 3.8 MB/s | 162 kB 00:00 2023-11-02T15:22:34.167 INFO:teuthology.orchestra.run.smithi175.stdout:(9/30): python3-rados-18.2.0-1159.gb5a9486b.el9 5.4 MB/s | 320 kB 00:00 2023-11-02T15:22:34.192 INFO:teuthology.orchestra.run.smithi174.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 343 kB/s | 49 kB 00:00 2023-11-02T15:22:34.226 INFO:teuthology.orchestra.run.smithi175.stdout:(10/30): python3-rbd-18.2.0-1159.gb5a9486b.el9. 5.0 MB/s | 298 kB 00:00 2023-11-02T15:22:34.276 INFO:teuthology.orchestra.run.smithi175.stdout:(11/30): python3-rgw-18.2.0-1159.gb5a9486b.el9. 1.9 MB/s | 99 kB 00:00 2023-11-02T15:22:34.334 INFO:teuthology.orchestra.run.smithi174.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 476 kB/s | 67 kB 00:00 2023-11-02T15:22:34.376 INFO:teuthology.orchestra.run.smithi175.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 1.0 MB/s | 106 kB 00:00 2023-11-02T15:22:34.418 INFO:teuthology.orchestra.run.smithi175.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2023-11-02T15:22:34.485 INFO:teuthology.orchestra.run.smithi175.stdout:(14/30): ceph-base-18.2.0-1159.gb5a9486b.el9.x8 2.1 MB/s | 5.2 MB 00:02 2023-11-02T15:22:34.511 INFO:teuthology.orchestra.run.smithi175.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2023-11-02T15:22:34.518 INFO:teuthology.orchestra.run.smithi174.stdout:(24/30): libarrow-9.0.0-9.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2023-11-02T15:22:34.544 INFO:teuthology.orchestra.run.smithi175.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-02T15:22:34.570 INFO:teuthology.orchestra.run.smithi175.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 710 kB/s | 41 kB 00:00 2023-11-02T15:22:34.652 INFO:teuthology.orchestra.run.smithi174.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-11-02T15:22:34.685 INFO:teuthology.orchestra.run.smithi174.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.3 MB/s | 837 kB 00:00 2023-11-02T15:22:34.729 INFO:teuthology.orchestra.run.smithi175.stdout:(18/30): ceph-radosgw-18.2.0-1159.gb5a9486b.el9 2.8 MB/s | 7.7 MB 00:02 2023-11-02T15:22:34.754 INFO:teuthology.orchestra.run.smithi175.stdout:(19/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2023-11-02T15:22:34.770 INFO:teuthology.orchestra.run.smithi175.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 552 kB/s | 111 kB 00:00 2023-11-02T15:22:34.827 INFO:teuthology.orchestra.run.smithi174.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2023-11-02T15:22:34.863 INFO:teuthology.orchestra.run.smithi175.stdout:(21/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 265 kB/s | 24 kB 00:00 2023-11-02T15:22:34.921 INFO:teuthology.orchestra.run.smithi175.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2023-11-02T15:22:34.938 INFO:teuthology.orchestra.run.smithi175.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2023-11-02T15:22:34.997 INFO:teuthology.orchestra.run.smithi175.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2023-11-02T15:22:35.047 INFO:teuthology.orchestra.run.smithi175.stdout:(25/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 7.5 MB/s | 837 kB 00:00 2023-11-02T15:22:35.127 INFO:teuthology.orchestra.run.smithi174.stdout:(28/30): librbd1-18.2.0-1159.gb5a9486b.el9.x86_ 10 MB/s | 3.0 MB 00:00 2023-11-02T15:22:35.130 INFO:teuthology.orchestra.run.smithi175.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-11-02T15:22:35.181 INFO:teuthology.orchestra.run.smithi175.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2023-11-02T15:22:35.249 INFO:teuthology.orchestra.run.smithi175.stdout:(28/30): libarrow-9.0.0-9.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2023-11-02T15:22:35.352 INFO:teuthology.orchestra.run.smithi174.stdout:(29/30): librados2-18.2.0-1159.gb5a9486b.el9.x8 4.9 MB/s | 3.3 MB 00:00 2023-11-02T15:22:35.464 INFO:teuthology.orchestra.run.smithi175.stdout:(29/30): librbd1-18.2.0-1159.gb5a9486b.el9.x86_ 11 MB/s | 3.0 MB 00:00 2023-11-02T15:22:35.740 INFO:teuthology.orchestra.run.smithi175.stdout:(30/30): librados2-18.2.0-1159.gb5a9486b.el9.x8 5.4 MB/s | 3.3 MB 00:00 2023-11-02T15:22:35.744 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:22:35.744 INFO:teuthology.orchestra.run.smithi175.stdout:Total 13 MB/s | 53 MB 00:03 2023-11-02T15:22:35.757 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:22:35.795 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:22:35.795 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:22:36.252 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:22:36.253 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:22:36.728 INFO:teuthology.orchestra.run.smithi174.stdout:(30/30): ceph-common-18.2.0-1159.gb5a9486b.el9. 3.4 MB/s | 18 MB 00:05 2023-11-02T15:22:36.731 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:22:36.731 INFO:teuthology.orchestra.run.smithi174.stdout:Total 9.4 MB/s | 53 MB 00:05 2023-11-02T15:22:36.744 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:22:36.782 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:22:36.783 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:22:37.252 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:22:37.253 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:22:37.624 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:22:37.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-11-02T15:22:37.750 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-11-02T15:22:37.820 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-11-02T15:22:38.130 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-11-02T15:22:38.202 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:38.294 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:38.383 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/32 2023-11-02T15:22:38.412 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:38.466 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:38.495 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-11-02T15:22:38.547 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9. 9/32 2023-11-02T15:22:38.608 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:22:38.613 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 10/32 2023-11-02T15:22:38.631 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 11/32 2023-11-02T15:22:38.665 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-11-02T15:22:38.680 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 11/32 2023-11-02T15:22:38.705 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-11-02T15:22:38.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-11-02T15:22:38.786 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-11-02T15:22:38.822 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-11-02T15:22:38.863 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-11-02T15:22:38.895 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-11-02T15:22:38.921 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-11-02T15:22:38.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-11-02T15:22:38.990 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-11-02T15:22:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-11-02T15:22:39.131 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:39.212 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-11-02T15:22:39.301 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/32 2023-11-02T15:22:39.331 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:39.338 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 20/32 2023-11-02T15:22:39.388 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:39.412 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 20/32 2023-11-02T15:22:39.422 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-11-02T15:22:39.443 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 21/32 2023-11-02T15:22:39.466 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9. 9/32 2023-11-02T15:22:39.470 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-11-02T15:22:39.531 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 10/32 2023-11-02T15:22:39.558 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 11/32 2023-11-02T15:22:39.607 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 11/32 2023-11-02T15:22:39.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-11-02T15:22:39.749 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-11-02T15:22:39.798 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-11-02T15:22:39.830 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-11-02T15:22:39.857 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-11-02T15:22:39.863 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-11-02T15:22:39.891 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 24/32 2023-11-02T15:22:39.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-11-02T15:22:39.926 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-11-02T15:22:39.963 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 24/32 2023-11-02T15:22:40.037 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 25/32 2023-11-02T15:22:40.068 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x8 26/32 2023-11-02T15:22:40.223 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-11-02T15:22:40.247 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 20/32 2023-11-02T15:22:40.336 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 20/32 2023-11-02T15:22:40.369 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 21/32 2023-11-02T15:22:40.404 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-11-02T15:22:40.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-11-02T15:22:40.842 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 24/32 2023-11-02T15:22:40.912 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 24/32 2023-11-02T15:22:40.979 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 25/32 2023-11-02T15:22:41.010 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x8 26/32 2023-11-02T15:22:42.045 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:42.082 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:42.635 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 28/32 2023-11-02T15:22:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 28/32 2023-11-02T15:22:42.764 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-02T15:22:42.764 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-02T15:22:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:42.797 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:43.042 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:43.082 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:43.624 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 27/32 2023-11-02T15:22:43.660 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 28/32 2023-11-02T15:22:43.738 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 28/32 2023-11-02T15:22:43.739 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-02T15:22:43.739 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-02T15:22:43.739 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:43.778 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:52.356 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:52.356 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2023-11-02T15:22:52.357 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2023-11-02T15:22:52.357 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2023-11-02T15:22:52.357 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2023-11-02T15:22:52.357 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2023-11-02T15:22:52.358 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2023-11-02T15:22:52.358 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2023-11-02T15:22:52.358 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/32 2023-11-02T15:22:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/32 2023-11-02T15:22:53.124 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:22:53.124 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-02T15:22:53.124 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-02T15:22:53.124 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-02T15:22:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-11-02T15:22:53.169 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:53.169 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2023-11-02T15:22:53.170 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2023-11-02T15:22:53.170 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2023-11-02T15:22:53.170 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2023-11-02T15:22:53.170 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2023-11-02T15:22:53.171 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2023-11-02T15:22:53.171 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2023-11-02T15:22:53.171 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:53.210 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-11-02T15:22:53.210 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:53.934 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/32 2023-11-02T15:22:53.958 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/32 2023-11-02T15:22:53.958 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:22:53.958 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-02T15:22:53.959 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-02T15:22:53.959 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-02T15:22:53.959 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:53.960 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-11-02T15:22:54.028 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-11-02T15:22:54.028 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/32 2023-11-02T15:22:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/32 2023-11-02T15:22:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 3/32 2023-11-02T15:22:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 4/32 2023-11-02T15:22:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 6/32 2023-11-02T15:22:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9. 8/32 2023-11-02T15:22:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x8 9/32 2023-11-02T15:22:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 10/32 2023-11-02T15:22:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 11/32 2023-11-02T15:22:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 12/32 2023-11-02T15:22:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 13/32 2023-11-02T15:22:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-11-02T15:22:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-11-02T15:22:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-11-02T15:22:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-11-02T15:22:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-11-02T15:22:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-11-02T15:22:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-11-02T15:22:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-11-02T15:22:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-11-02T15:22:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-11-02T15:22:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-11-02T15:22:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-11-02T15:22:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-11-02T15:22:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-11-02T15:22:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-11-02T15:22:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-11-02T15:22:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/32 2023-11-02T15:22:55.105 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:55.105 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/32 2023-11-02T15:22:55.106 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/32 2023-11-02T15:22:55.106 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 3/32 2023-11-02T15:22:55.106 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 4/32 2023-11-02T15:22:55.107 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/32 2023-11-02T15:22:55.107 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_6 6/32 2023-11-02T15:22:55.107 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 7/32 2023-11-02T15:22:55.108 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9. 8/32 2023-11-02T15:22:55.108 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x8 9/32 2023-11-02T15:22:55.108 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 10/32 2023-11-02T15:22:55.109 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 11/32 2023-11-02T15:22:55.109 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 12/32 2023-11-02T15:22:55.109 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 13/32 2023-11-02T15:22:55.110 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-11-02T15:22:55.110 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-11-02T15:22:55.110 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-11-02T15:22:55.111 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-11-02T15:22:55.111 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-11-02T15:22:55.111 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-11-02T15:22:55.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-11-02T15:22:55.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-11-02T15:22:55.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-11-02T15:22:55.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-11-02T15:22:55.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-11-02T15:22:55.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-11-02T15:22:55.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-11-02T15:22:55.115 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-11-02T15:22:55.115 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-11-02T15:22:55.118 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/32 2023-11-02T15:22:55.118 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-11-02T15:22:55.119 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/32 2023-11-02T15:22:55.242 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:55.243 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:55.243 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2023-11-02T15:22:55.243 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.243 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.244 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:22:55.244 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-02T15:22:55.244 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.244 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.245 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.245 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.246 INFO:teuthology.orchestra.run.smithi175.stdout: fmt-8.1.1-5.el9.x86_64 2023-11-02T15:22:55.246 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-02T15:22:55.246 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-02T15:22:55.247 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-02T15:22:55.247 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.247 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-02T15:22:55.249 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-02T15:22:55.249 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-02T15:22:55.250 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.250 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-02T15:22:55.250 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.250 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-02T15:22:55.250 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-02T15:22:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-02T15:22:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-02T15:22:55.252 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.252 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.252 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:55.252 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2023-11-02T15:22:55.252 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-02T15:22:55.253 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:55.253 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:22:55.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test 2023-11-02T15:22:56.109 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:27 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:22:56.256 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:22:56.257 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:56.257 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:22:56.257 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:56.257 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:22:56.258 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 40 M 2023-11-02T15:22:56.258 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:22:56.258 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-11-02T15:22:56.258 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 166 k 2023-11-02T15:22:56.258 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-02T15:22:56.259 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-02T15:22:56.259 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-02T15:22:56.259 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:56.259 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:22:56.259 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:22:56.260 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2023-11-02T15:22:56.260 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:22:56.260 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 41 M 2023-11-02T15:22:56.261 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 167 M 2023-11-02T15:22:56.261 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:22:56.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-11-02T15:22:56.324 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:56.324 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2023-11-02T15:22:56.325 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.325 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.325 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:22:56.325 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-02T15:22:56.326 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.326 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.326 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.326 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.326 INFO:teuthology.orchestra.run.smithi174.stdout: fmt-8.1.1-5.el9.x86_64 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-02T15:22:56.327 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-02T15:22:56.328 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-02T15:22:56.328 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.328 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-02T15:22:56.328 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.328 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-02T15:22:56.329 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-02T15:22:56.329 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-02T15:22:56.329 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.329 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.329 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.330 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-02T15:22:56.330 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.330 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.330 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:22:56.330 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2023-11-02T15:22:56.331 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-02T15:22:56.331 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:56.331 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:22:56.539 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.5 MB/s | 217 kB 00:00 2023-11-02T15:22:56.581 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 300 kB 00:00 2023-11-02T15:22:56.615 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-11-02T15:22:56.657 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): libcephsqlite-18.2.0-1159.gb5a9486b.el9. 635 kB/s | 166 kB 00:00 2023-11-02T15:22:56.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-test 2023-11-02T15:22:57.323 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:29 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:22:57.398 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): jq-1.6-15.el9.x86_64.rpm 239 kB/s | 187 kB 00:00 2023-11-02T15:22:57.467 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:22:57.468 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:57.468 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:22:57.468 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:57.468 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:22:57.469 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 40 M 2023-11-02T15:22:57.469 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:22:57.469 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-11-02T15:22:57.469 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 166 k 2023-11-02T15:22:57.469 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-02T15:22:57.470 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-02T15:22:57.470 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-02T15:22:57.470 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:57.470 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:22:57.471 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:22:57.471 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2023-11-02T15:22:57.471 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:22:57.472 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 41 M 2023-11-02T15:22:57.472 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 167 M 2023-11-02T15:22:57.472 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:22:57.899 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.4 MB/s | 217 kB 00:00 2023-11-02T15:22:57.941 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 300 kB 00:00 2023-11-02T15:22:57.965 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2023-11-02T15:22:58.091 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): jq-1.6-15.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-11-02T15:22:58.183 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): libcephsqlite-18.2.0-1159.gb5a9486b.el9. 382 kB/s | 166 kB 00:00 2023-11-02T15:22:58.815 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): ceph-test-18.2.0-1159.gb5a9486b.el9.x86_ 16 MB/s | 40 MB 00:02 2023-11-02T15:22:58.818 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:22:58.819 INFO:teuthology.orchestra.run.smithi175.stdout:Total 16 MB/s | 41 MB 00:02 2023-11-02T15:22:58.833 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:22:58.853 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:22:58.853 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:22:59.193 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:22:59.194 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:22:59.730 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:22:59.812 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-11-02T15:22:59.855 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-11-02T15:22:59.906 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-11-02T15:22:59.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-11-02T15:22:59.976 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/6 2023-11-02T15:23:00.925 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): ceph-test-18.2.0-1159.gb5a9486b.el9.x86_ 12 MB/s | 40 MB 00:03 2023-11-02T15:23:00.927 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:00.928 INFO:teuthology.orchestra.run.smithi174.stdout:Total 12 MB/s | 41 MB 00:03 2023-11-02T15:23:00.942 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:00.963 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:00.963 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:01.305 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:01.305 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:01.838 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:01.912 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-11-02T15:23:01.953 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-11-02T15:23:01.982 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-11-02T15:23:02.025 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-11-02T15:23:02.052 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/6 2023-11-02T15:23:03.741 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/6 2023-11-02T15:23:03.809 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/6 2023-11-02T15:23:05.804 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/6 2023-11-02T15:23:05.875 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/6 2023-11-02T15:23:05.891 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/6 2023-11-02T15:23:05.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/6 2023-11-02T15:23:05.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/6 2023-11-02T15:23:05.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-11-02T15:23:05.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-11-02T15:23:05.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-11-02T15:23:06.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-11-02T15:23:06.286 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:06.286 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:06.286 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:06.287 INFO:teuthology.orchestra.run.smithi175.stdout: jq-1.6-15.el9.x86_64 2023-11-02T15:23:06.287 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:06.287 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-02T15:23:06.287 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-02T15:23:06.287 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-02T15:23:06.288 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:06.288 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:06.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph 2023-11-02T15:23:07.113 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:38 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:07.258 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:07.260 INFO:teuthology.orchestra.run.smithi175.stdout:========================================================================================= 2023-11-02T15:23:07.260 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:07.261 INFO:teuthology.orchestra.run.smithi175.stdout:========================================================================================= 2023-11-02T15:23:07.261 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:07.261 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 6.5 k 2023-11-02T15:23:07.261 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:07.261 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 2.1 M 2023-11-02T15:23:07.262 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.4 M 2023-11-02T15:23:07.262 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 242 k 2023-11-02T15:23:07.262 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.4 M 2023-11-02T15:23:07.262 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 18 M 2023-11-02T15:23:07.262 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-02T15:23:07.263 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-02T15:23:07.263 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-02T15:23:07.263 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-02T15:23:07.263 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-02T15:23:07.263 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-02T15:23:07.264 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-02T15:23:07.264 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-02T15:23:07.264 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-02T15:23:07.264 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-02T15:23:07.264 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-02T15:23:07.265 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-02T15:23:07.266 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-02T15:23:07.266 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-02T15:23:07.266 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-02T15:23:07.266 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-02T15:23:07.266 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-02T15:23:07.267 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:07.268 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:07.268 INFO:teuthology.orchestra.run.smithi175.stdout:========================================================================================= 2023-11-02T15:23:07.268 INFO:teuthology.orchestra.run.smithi175.stdout:Install 33 Packages 2023-11-02T15:23:07.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:07.269 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 30 M 2023-11-02T15:23:07.269 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 109 M 2023-11-02T15:23:07.269 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:07.929 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 6/6 2023-11-02T15:23:07.929 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/6 2023-11-02T15:23:07.929 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/6 2023-11-02T15:23:07.930 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-11-02T15:23:07.930 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-11-02T15:23:07.930 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-11-02T15:23:08.075 INFO:teuthology.orchestra.run.smithi175.stdout:(1/33): ceph-18.2.0-1159.gb5a9486b.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2023-11-02T15:23:08.337 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-11-02T15:23:08.337 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:08.337 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:08.338 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:08.338 INFO:teuthology.orchestra.run.smithi174.stdout: jq-1.6-15.el9.x86_64 2023-11-02T15:23:08.338 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:08.338 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-02T15:23:08.339 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-02T15:23:08.339 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-02T15:23:08.339 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:08.339 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:08.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph 2023-11-02T15:23:08.783 INFO:teuthology.orchestra.run.smithi175.stdout:(2/33): ceph-mgr-18.2.0-1159.gb5a9486b.el9.x86_ 1.4 MB/s | 1.4 MB 00:01 2023-11-02T15:23:08.833 INFO:teuthology.orchestra.run.smithi175.stdout:(3/33): ceph-mds-18.2.0-1159.gb5a9486b.el9.x86_ 1.9 MB/s | 2.1 MB 00:01 2023-11-02T15:23:08.967 INFO:teuthology.orchestra.run.smithi175.stdout:(4/33): ceph-mgr-modules-core-18.2.0-1159.gb5a9 1.8 MB/s | 242 kB 00:00 2023-11-02T15:23:09.067 INFO:teuthology.orchestra.run.smithi175.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2023-11-02T15:23:09.100 INFO:teuthology.orchestra.run.smithi175.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 945 kB/s | 32 kB 00:00 2023-11-02T15:23:09.134 INFO:teuthology.orchestra.run.smithi175.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-11-02T15:23:09.176 INFO:teuthology.orchestra.run.smithi175.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2023-11-02T15:23:09.185 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:41 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:09.235 INFO:teuthology.orchestra.run.smithi175.stdout:(9/33): ceph-mon-18.2.0-1159.gb5a9486b.el9.x86_ 3.8 MB/s | 4.4 MB 00:01 2023-11-02T15:23:09.260 INFO:teuthology.orchestra.run.smithi175.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-11-02T15:23:09.277 INFO:teuthology.orchestra.run.smithi175.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2023-11-02T15:23:09.302 INFO:teuthology.orchestra.run.smithi175.stdout:(12/33): python3-requests-2.25.1-7.el9.noarch.r 2.6 MB/s | 113 kB 00:00 2023-11-02T15:23:09.327 INFO:teuthology.orchestra.run.smithi175.stdout:(13/33): python3-urllib3-1.26.5-4.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2023-11-02T15:23:09.335 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:09.337 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2023-11-02T15:23:09.337 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:09.338 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2023-11-02T15:23:09.338 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:09.338 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 6.5 k 2023-11-02T15:23:09.339 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:09.339 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 2.1 M 2023-11-02T15:23:09.339 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.4 M 2023-11-02T15:23:09.339 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 242 k 2023-11-02T15:23:09.339 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.4 M 2023-11-02T15:23:09.340 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 18 M 2023-11-02T15:23:09.340 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-02T15:23:09.340 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-02T15:23:09.340 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-02T15:23:09.341 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-02T15:23:09.341 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-02T15:23:09.341 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-02T15:23:09.341 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-02T15:23:09.341 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-02T15:23:09.342 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-02T15:23:09.342 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-02T15:23:09.342 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-02T15:23:09.342 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-02T15:23:09.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-02T15:23:09.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-02T15:23:09.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-02T15:23:09.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-02T15:23:09.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-02T15:23:09.344 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-02T15:23:09.344 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-02T15:23:09.344 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-02T15:23:09.344 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-02T15:23:09.345 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-02T15:23:09.345 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-02T15:23:09.345 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-02T15:23:09.345 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-02T15:23:09.346 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-02T15:23:09.346 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-02T15:23:09.346 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:09.346 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:09.346 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2023-11-02T15:23:09.347 INFO:teuthology.orchestra.run.smithi174.stdout:Install 33 Packages 2023-11-02T15:23:09.347 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 30 M 2023-11-02T15:23:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 109 M 2023-11-02T15:23:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:09.478 INFO:teuthology.orchestra.run.smithi175.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 340 kB/s | 51 kB 00:00 2023-11-02T15:23:09.811 INFO:teuthology.orchestra.run.smithi175.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 130 kB/s | 43 kB 00:00 2023-11-02T15:23:10.017 INFO:teuthology.orchestra.run.smithi174.stdout:(1/33): ceph-18.2.0-1159.gb5a9486b.el9.x86_64.r 16 kB/s | 6.5 kB 00:00 2023-11-02T15:23:10.178 INFO:teuthology.orchestra.run.smithi175.stdout:(16/33): ceph-osd-18.2.0-1159.gb5a9486b.el9.x86 13 MB/s | 18 MB 00:01 2023-11-02T15:23:10.203 INFO:teuthology.orchestra.run.smithi175.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 438 kB/s | 172 kB 00:00 2023-11-02T15:23:10.234 INFO:teuthology.orchestra.run.smithi174.stdout:(2/33): ceph-mgr-18.2.0-1159.gb5a9486b.el9.x86_ 2.3 MB/s | 1.4 MB 00:00 2023-11-02T15:23:10.237 INFO:teuthology.orchestra.run.smithi175.stdout:(18/33): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2023-11-02T15:23:10.304 INFO:teuthology.orchestra.run.smithi175.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2023-11-02T15:23:10.376 INFO:teuthology.orchestra.run.smithi174.stdout:(3/33): ceph-mds-18.2.0-1159.gb5a9486b.el9.x86_ 2.7 MB/s | 2.1 MB 00:00 2023-11-02T15:23:10.413 INFO:teuthology.orchestra.run.smithi175.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 214 kB/s | 23 kB 00:00 2023-11-02T15:23:10.460 INFO:teuthology.orchestra.run.smithi174.stdout:(4/33): ceph-mgr-modules-core-18.2.0-1159.gb5a9 2.8 MB/s | 242 kB 00:00 2023-11-02T15:23:10.479 INFO:teuthology.orchestra.run.smithi175.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 73 kB/s | 18 kB 00:00 2023-11-02T15:23:10.513 INFO:teuthology.orchestra.run.smithi175.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2023-11-02T15:23:10.560 INFO:teuthology.orchestra.run.smithi174.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2023-11-02T15:23:10.593 INFO:teuthology.orchestra.run.smithi174.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 948 kB/s | 32 kB 00:00 2023-11-02T15:23:10.613 INFO:teuthology.orchestra.run.smithi175.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 147 kB/s | 20 kB 00:00 2023-11-02T15:23:10.627 INFO:teuthology.orchestra.run.smithi174.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-11-02T15:23:10.638 INFO:teuthology.orchestra.run.smithi175.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 370 kB/s | 46 kB 00:00 2023-11-02T15:23:10.669 INFO:teuthology.orchestra.run.smithi174.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2023-11-02T15:23:10.703 INFO:teuthology.orchestra.run.smithi174.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2023-11-02T15:23:10.705 INFO:teuthology.orchestra.run.smithi175.stdout:(25/33): python3-cherrypy-18.6.1-2.el9.noarch.r 680 kB/s | 358 kB 00:00 2023-11-02T15:23:10.736 INFO:teuthology.orchestra.run.smithi174.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 3.7 MB/s | 124 kB 00:00 2023-11-02T15:23:10.770 INFO:teuthology.orchestra.run.smithi174.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2023-11-02T15:23:10.804 INFO:teuthology.orchestra.run.smithi174.stdout:(12/33): python3-urllib3-1.26.5-4.el9.noarch.rp 5.4 MB/s | 187 kB 00:00 2023-11-02T15:23:10.839 INFO:teuthology.orchestra.run.smithi175.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2023-11-02T15:23:10.864 INFO:teuthology.orchestra.run.smithi175.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 314 kB/s | 79 kB 00:00 2023-11-02T15:23:10.889 INFO:teuthology.orchestra.run.smithi175.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 89 kB/s | 16 kB 00:00 2023-11-02T15:23:10.940 INFO:teuthology.orchestra.run.smithi175.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 896 kB/s | 90 kB 00:00 2023-11-02T15:23:10.982 INFO:teuthology.orchestra.run.smithi175.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 305 kB/s | 36 kB 00:00 2023-11-02T15:23:11.082 INFO:teuthology.orchestra.run.smithi175.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2023-11-02T15:23:11.107 INFO:teuthology.orchestra.run.smithi175.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2023-11-02T15:23:11.124 INFO:teuthology.orchestra.run.smithi175.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 981 kB/s | 230 kB 00:00 2023-11-02T15:23:11.126 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:11.127 INFO:teuthology.orchestra.run.smithi175.stdout:Total 7.7 MB/s | 30 MB 00:03 2023-11-02T15:23:11.160 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:11.199 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:11.199 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:11.562 INFO:teuthology.orchestra.run.smithi174.stdout:(13/33): python3-cryptography-36.0.1-4.el9.x86_ 1.6 MB/s | 1.2 MB 00:00 2023-11-02T15:23:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:11.662 INFO:teuthology.orchestra.run.smithi174.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 512 kB/s | 51 kB 00:00 2023-11-02T15:23:12.054 INFO:teuthology.orchestra.run.smithi174.stdout:(15/33): ceph-osd-18.2.0-1159.gb5a9486b.el9.x86 9.7 MB/s | 18 MB 00:01 2023-11-02T15:23:12.079 INFO:teuthology.orchestra.run.smithi174.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 104 kB/s | 43 kB 00:00 2023-11-02T15:23:12.346 INFO:teuthology.orchestra.run.smithi174.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2023-11-02T15:23:12.438 INFO:teuthology.orchestra.run.smithi174.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2023-11-02T15:23:12.463 INFO:teuthology.orchestra.run.smithi174.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 420 kB/s | 172 kB 00:00 2023-11-02T15:23:12.506 INFO:teuthology.orchestra.run.smithi174.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 265 kB/s | 18 kB 00:00 2023-11-02T15:23:12.531 INFO:teuthology.orchestra.run.smithi174.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2023-11-02T15:23:12.573 INFO:teuthology.orchestra.run.smithi174.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 291 kB/s | 19 kB 00:00 2023-11-02T15:23:12.598 INFO:teuthology.orchestra.run.smithi174.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 293 kB/s | 20 kB 00:00 2023-11-02T15:23:12.665 INFO:teuthology.orchestra.run.smithi174.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 504 kB/s | 46 kB 00:00 2023-11-02T15:23:12.699 INFO:teuthology.orchestra.run.smithi174.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 787 kB/s | 79 kB 00:00 2023-11-02T15:23:12.799 INFO:teuthology.orchestra.run.smithi174.stdout:(26/33): ceph-mon-18.2.0-1159.gb5a9486b.el9.x86 1.6 MB/s | 4.4 MB 00:02 2023-11-02T15:23:12.812 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:12.824 INFO:teuthology.orchestra.run.smithi174.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2023-11-02T15:23:12.858 INFO:teuthology.orchestra.run.smithi174.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2023-11-02T15:23:12.868 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-11-02T15:23:12.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-11-02T15:23:12.916 INFO:teuthology.orchestra.run.smithi174.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 390 kB/s | 36 kB 00:00 2023-11-02T15:23:12.942 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-11-02T15:23:13.016 INFO:teuthology.orchestra.run.smithi174.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-11-02T15:23:13.033 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-11-02T15:23:13.092 INFO:teuthology.orchestra.run.smithi174.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2023-11-02T15:23:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-11-02T15:23:13.117 INFO:teuthology.orchestra.run.smithi174.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2023-11-02T15:23:13.144 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-11-02T15:23:13.176 INFO:teuthology.orchestra.run.smithi174.stdout:(33/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 239 kB/s | 90 kB 00:00 2023-11-02T15:23:13.181 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:13.181 INFO:teuthology.orchestra.run.smithi174.stdout:Total 7.8 MB/s | 30 MB 00:03 2023-11-02T15:23:13.182 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-11-02T15:23:13.214 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:13.249 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-11-02T15:23:13.253 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:13.254 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:13.304 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 9/33 2023-11-02T15:23:13.342 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-11-02T15:23:13.391 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-11-02T15:23:13.454 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-11-02T15:23:13.614 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:13.614 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:13.646 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-11-02T15:23:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-11-02T15:23:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-11-02T15:23:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-11-02T15:23:13.799 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-11-02T15:23:13.883 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-11-02T15:23:13.916 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-11-02T15:23:13.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-11-02T15:23:13.991 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-11-02T15:23:14.026 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-11-02T15:23:14.116 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-11-02T15:23:14.156 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-11-02T15:23:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-11-02T15:23:14.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-11-02T15:23:14.500 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-11-02T15:23:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9. 28/33 2023-11-02T15:23:14.678 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/33 2023-11-02T15:23:14.702 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/33 2023-11-02T15:23:14.702 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:14.703 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-02T15:23:14.703 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-02T15:23:14.703 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-02T15:23:14.703 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:14.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-11-02T15:23:14.960 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-11-02T15:23:14.993 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-11-02T15:23:15.082 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-11-02T15:23:15.160 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-11-02T15:23:15.201 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-11-02T15:23:15.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-11-02T15:23:15.324 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-11-02T15:23:15.377 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 9/33 2023-11-02T15:23:15.416 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-11-02T15:23:15.465 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-11-02T15:23:15.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-11-02T15:23:15.741 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-11-02T15:23:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-11-02T15:23:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-11-02T15:23:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-11-02T15:23:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-11-02T15:23:15.966 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-11-02T15:23:16.008 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-11-02T15:23:16.042 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-11-02T15:23:16.083 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-11-02T15:23:16.117 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-11-02T15:23:16.203 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/33 2023-11-02T15:23:16.206 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-11-02T15:23:16.229 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/33 2023-11-02T15:23:16.229 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:16.229 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-02T15:23:16.230 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-02T15:23:16.230 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-02T15:23:16.230 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:16.248 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-11-02T15:23:16.321 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-11-02T15:23:16.457 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-11-02T15:23:16.583 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-11-02T15:23:16.679 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/33 2023-11-02T15:23:16.702 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/33 2023-11-02T15:23:16.703 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:16.703 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-02T15:23:16.703 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-02T15:23:16.703 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-02T15:23:16.703 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:16.746 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9. 28/33 2023-11-02T15:23:16.778 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/33 2023-11-02T15:23:16.802 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 29/33 2023-11-02T15:23:16.802 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:16.802 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-02T15:23:16.802 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-02T15:23:16.803 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-02T15:23:16.803 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:16.904 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 32/33 2023-11-02T15:23:16.927 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 32/33 2023-11-02T15:23:16.928 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:16.928 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-02T15:23:16.928 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-02T15:23:16.928 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-02T15:23:16.928 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 33/33 2023-11-02T15:23:18.304 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/33 2023-11-02T15:23:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 30/33 2023-11-02T15:23:18.330 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:18.331 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-02T15:23:18.331 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-02T15:23:18.331 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-02T15:23:18.331 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:18.700 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 33/33 2023-11-02T15:23:18.701 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/33 2023-11-02T15:23:18.701 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/33 2023-11-02T15:23:18.701 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 3/33 2023-11-02T15:23:18.702 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 4/33 2023-11-02T15:23:18.702 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/33 2023-11-02T15:23:18.702 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9. 6/33 2023-11-02T15:23:18.702 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-11-02T15:23:18.702 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-11-02T15:23:18.703 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-11-02T15:23:18.703 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-11-02T15:23:18.703 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-11-02T15:23:18.703 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-11-02T15:23:18.704 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-11-02T15:23:18.704 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 14/33 2023-11-02T15:23:18.704 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-11-02T15:23:18.704 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-11-02T15:23:18.704 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-11-02T15:23:18.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-11-02T15:23:18.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-11-02T15:23:18.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-11-02T15:23:18.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-11-02T15:23:18.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-11-02T15:23:18.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-11-02T15:23:18.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-11-02T15:23:18.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-11-02T15:23:18.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-11-02T15:23:18.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-11-02T15:23:18.707 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-11-02T15:23:18.707 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-11-02T15:23:18.708 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-11-02T15:23:18.708 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-11-02T15:23:18.708 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-11-02T15:23:18.779 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/33 2023-11-02T15:23:18.802 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 31/33 2023-11-02T15:23:18.802 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:18.802 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-02T15:23:18.803 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-02T15:23:18.803 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-02T15:23:18.803 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:19.005 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 32/33 2023-11-02T15:23:19.027 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 32/33 2023-11-02T15:23:19.028 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:19.028 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-02T15:23:19.028 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-02T15:23:19.028 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-02T15:23:19.029 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:19.093 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 33/33 2023-11-02T15:23:19.694 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-11-02T15:23:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:19.695 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-02T15:23:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-02T15:23:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-02T15:23:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-02T15:23:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-02T15:23:19.697 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-02T15:23:19.698 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-02T15:23:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-02T15:23:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-02T15:23:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-02T15:23:19.699 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-02T15:23:19.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-02T15:23:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-02T15:23:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-02T15:23:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-02T15:23:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-02T15:23:19.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-02T15:23:19.702 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:19.702 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:20.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base 2023-11-02T15:23:20.560 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:51 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:20.650 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:23:20.698 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:20.699 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:23:20.700 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:20.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install cephadm 2023-11-02T15:23:20.807 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 33/33 2023-11-02T15:23:20.807 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/33 2023-11-02T15:23:20.808 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/33 2023-11-02T15:23:20.808 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 3/33 2023-11-02T15:23:20.808 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 4/33 2023-11-02T15:23:20.808 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 5/33 2023-11-02T15:23:20.808 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9. 6/33 2023-11-02T15:23:20.809 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-11-02T15:23:20.809 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-11-02T15:23:20.809 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-11-02T15:23:20.809 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-11-02T15:23:20.809 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-11-02T15:23:20.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-11-02T15:23:20.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-11-02T15:23:20.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 14/33 2023-11-02T15:23:20.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-11-02T15:23:20.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-11-02T15:23:20.811 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-11-02T15:23:20.811 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-11-02T15:23:20.811 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-11-02T15:23:20.811 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-11-02T15:23:20.811 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-11-02T15:23:20.812 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-11-02T15:23:20.812 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-11-02T15:23:20.812 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-11-02T15:23:20.812 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-11-02T15:23:20.812 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-11-02T15:23:20.813 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-11-02T15:23:20.813 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-11-02T15:23:20.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-11-02T15:23:20.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-11-02T15:23:20.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-11-02T15:23:20.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-11-02T15:23:21.317 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:52 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:21.466 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:21.467 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:21.467 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:21.467 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:21.467 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:21.468 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 221 k 2023-11-02T15:23:21.468 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:21.468 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:21.468 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:21.468 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:23:21.469 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:21.469 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 221 k 2023-11-02T15:23:21.469 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 213 k 2023-11-02T15:23:21.469 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:21.665 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-11-02T15:23:21.666 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:21.666 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:21.666 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:21.667 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:21.667 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:21.667 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:21.667 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:21.667 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:21.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-02T15:23:21.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-02T15:23:21.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-02T15:23:21.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-02T15:23:21.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-02T15:23:21.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-02T15:23:21.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-02T15:23:21.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-02T15:23:21.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-02T15:23:21.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-02T15:23:21.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-02T15:23:21.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-02T15:23:21.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-02T15:23:21.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-02T15:23:21.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-02T15:23:21.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2023-11-02T15:23:21.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-02T15:23:21.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-02T15:23:21.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-02T15:23:21.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-02T15:23:21.672 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-02T15:23:21.673 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-02T15:23:21.673 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:21.673 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:22.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-base 2023-11-02T15:23:22.004 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm-18.2.0-1159.gb5a9486b.el9.noarch.rpm 414 kB/s | 221 kB 00:00 2023-11-02T15:23:22.005 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:22.005 INFO:teuthology.orchestra.run.smithi175.stdout:Total 412 kB/s | 221 kB 00:00 2023-11-02T15:23:22.005 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:22.009 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:22.010 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:22.012 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:22.012 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:22.546 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:22.575 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:54 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:22.640 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:22.665 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-base-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:23:22.713 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:22.714 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:23:22.715 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:22.768 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install cephadm 2023-11-02T15:23:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:23.301 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:55 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:23.307 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:23.307 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:23.307 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:23.308 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:23.308 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:23.308 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:23.441 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:23.442 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:23.442 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:23.442 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:23.442 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:23.442 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 221 k 2023-11-02T15:23:23.443 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:23.443 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:23.443 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:23.443 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:23:23.443 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:23.444 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 221 k 2023-11-02T15:23:23.444 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 213 k 2023-11-02T15:23:23.444 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:23.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache 2023-11-02T15:23:23.802 INFO:teuthology.orchestra.run.smithi174.stdout:cephadm-18.2.0-1159.gb5a9486b.el9.noarch.rpm 616 kB/s | 221 kB 00:00 2023-11-02T15:23:23.803 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:23.803 INFO:teuthology.orchestra.run.smithi174.stdout:Total 613 kB/s | 221 kB 00:00 2023-11-02T15:23:23.803 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:23.808 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:23.808 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:23.810 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:23.811 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:23.879 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:24.084 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:55 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:24.224 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:24.225 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:23:24.226 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:24.226 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 142 k 2023-11-02T15:23:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:24.226 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:24.227 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:24.227 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:23:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:24.227 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 142 k 2023-11-02T15:23:24.227 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 400 k 2023-11-02T15:23:24.228 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:24.253 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:24.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:24.550 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-18.2.0-1159.gb5a948 440 kB/s | 142 kB 00:00 2023-11-02T15:23:24.550 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:24.551 INFO:teuthology.orchestra.run.smithi175.stdout:Total 438 kB/s | 142 kB 00:00 2023-11-02T15:23:24.551 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:24.558 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:24.558 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:24.610 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:24.613 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:24.720 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:24.758 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:24.817 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:24.841 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:24.841 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:24.841 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-02T15:23:24.842 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:25.036 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:25.036 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:25.036 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:25.036 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:25.037 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:25.037 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:25.197 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-immutable-object-cache 2023-11-02T15:23:25.751 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:57 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:25.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:25.793 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:25.793 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:25.794 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:25.794 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:25.794 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:25.896 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:25.897 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:25.898 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:23:25.898 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:25.898 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:25.898 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 142 k 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:25.899 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 142 k 2023-11-02T15:23:25.900 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 400 k 2023-11-02T15:23:25.900 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:26.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr 2023-11-02T15:23:26.215 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-immutable-object-cache-18.2.0-1159.gb5a948 450 kB/s | 142 kB 00:00 2023-11-02T15:23:26.216 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:26.217 INFO:teuthology.orchestra.run.smithi174.stdout:Total 448 kB/s | 142 kB 00:00 2023-11-02T15:23:26.217 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:26.223 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:26.224 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:26.276 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:26.276 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:26.417 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:26.483 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:26.507 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:26.507 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:26.507 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-02T15:23:26.508 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:26.597 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:57 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:26.689 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:23:26.737 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:26.738 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:23:26.738 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:26.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-dashboard 2023-11-02T15:23:27.343 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:58 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:27.483 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:27.484 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:27.484 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:27.484 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 1.8 M 2023-11-02T15:23:27.485 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 23 k 2023-11-02T15:23:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 15 k 2023-11-02T15:23:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-02T15:23:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-02T15:23:27.486 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-02T15:23:27.486 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2023-11-02T15:23:27.486 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-02T15:23:27.486 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:27.486 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:27.487 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:27.487 INFO:teuthology.orchestra.run.smithi175.stdout:Install 7 Packages 2023-11-02T15:23:27.487 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:27.488 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.1 M 2023-11-02T15:23:27.488 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 8.5 M 2023-11-02T15:23:27.488 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:27.493 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b. 1/1 2023-11-02T15:23:27.494 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:27.494 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:27.494 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:23:27.494 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:27.494 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:27.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr 2023-11-02T15:23:27.744 INFO:teuthology.orchestra.run.smithi175.stdout:(1/7): ceph-grafana-dashboards-18.2.0-1159.gb5a 127 kB/s | 23 kB 00:00 2023-11-02T15:23:27.844 INFO:teuthology.orchestra.run.smithi175.stdout:(2/7): ceph-prometheus-alerts-18.2.0-1159.gb5a9 52 kB/s | 15 kB 00:00 2023-11-02T15:23:28.045 INFO:teuthology.orchestra.run.smithi175.stdout:(3/7): ceph-mgr-dashboard-18.2.0-1159.gb5a9486b 3.6 MB/s | 1.8 MB 00:00 2023-11-02T15:23:28.255 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:00 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:28.295 INFO:teuthology.orchestra.run.smithi175.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 16 kB/s | 9.0 kB 00:00 2023-11-02T15:23:28.337 INFO:teuthology.orchestra.run.smithi175.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 83 kB/s | 41 kB 00:00 2023-11-02T15:23:28.346 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-mgr-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:23:28.394 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:28.395 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:23:28.396 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:28.445 INFO:teuthology.orchestra.run.smithi175.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2023-11-02T15:23:28.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-dashboard 2023-11-02T15:23:28.998 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:00 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:29.140 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:29.140 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:29.141 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:29.141 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:29.141 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:29.141 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 1.8 M 2023-11-02T15:23:29.141 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:29.142 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 23 k 2023-11-02T15:23:29.142 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 15 k 2023-11-02T15:23:29.142 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-02T15:23:29.142 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-02T15:23:29.142 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-02T15:23:29.143 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2023-11-02T15:23:29.143 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-02T15:23:29.143 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:29.143 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:29.143 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:29.144 INFO:teuthology.orchestra.run.smithi174.stdout:Install 7 Packages 2023-11-02T15:23:29.144 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:29.144 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.1 M 2023-11-02T15:23:29.145 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 8.5 M 2023-11-02T15:23:29.145 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:29.237 INFO:teuthology.orchestra.run.smithi175.stdout:(7/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 26 kB/s | 31 kB 00:01 2023-11-02T15:23:29.239 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:29.240 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.2 MB/s | 2.1 MB 00:01 2023-11-02T15:23:29.240 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:29.249 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:29.249 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:29.323 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:29.324 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:29.385 INFO:teuthology.orchestra.run.smithi174.stdout:(1/7): ceph-prometheus-alerts-18.2.0-1159.gb5a9 77 kB/s | 15 kB 00:00 2023-11-02T15:23:29.435 INFO:teuthology.orchestra.run.smithi174.stdout:(2/7): ceph-grafana-dashboards-18.2.0-1159.gb5a 96 kB/s | 23 kB 00:00 2023-11-02T15:23:29.615 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:29.672 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-11-02T15:23:29.719 INFO:teuthology.orchestra.run.smithi174.stdout:(3/7): ceph-mgr-dashboard-18.2.0-1159.gb5a9486b 3.3 MB/s | 1.8 MB 00:00 2023-11-02T15:23:29.731 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-11-02T15:23:29.769 INFO:teuthology.orchestra.run.smithi174.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 24 kB/s | 9.0 kB 00:00 2023-11-02T15:23:29.777 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-11-02T15:23:29.817 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-11-02T15:23:29.838 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.n 5/7 2023-11-02T15:23:29.994 INFO:teuthology.orchestra.run.smithi174.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 73 kB/s | 41 kB 00:00 2023-11-02T15:23:30.072 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9. 6/7 2023-11-02T15:23:30.116 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 7/7 2023-11-02T15:23:30.119 INFO:teuthology.orchestra.run.smithi174.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 539 kB/s | 188 kB 00:00 2023-11-02T15:23:30.178 INFO:teuthology.orchestra.run.smithi174.stdout:(7/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 67 kB/s | 31 kB 00:00 2023-11-02T15:23:30.182 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:30.182 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.0 MB/s | 2.1 MB 00:01 2023-11-02T15:23:30.183 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:30.190 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:30.191 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:30.257 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:30.257 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:30.557 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:30.621 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-11-02T15:23:30.655 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-11-02T15:23:30.677 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 7/7 2023-11-02T15:23:30.678 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9. 1/7 2023-11-02T15:23:30.678 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 2/7 2023-11-02T15:23:30.678 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.n 3/7 2023-11-02T15:23:30.679 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-11-02T15:23:30.679 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-11-02T15:23:30.679 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-11-02T15:23:30.701 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-11-02T15:23:30.742 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-11-02T15:23:30.770 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.n 5/7 2023-11-02T15:23:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-11-02T15:23:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:30.973 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-02T15:23:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-02T15:23:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-02T15:23:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-02T15:23:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:30.975 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:31.006 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9. 6/7 2023-11-02T15:23:31.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 7/7 2023-11-02T15:23:31.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-diskprediction-local 2023-11-02T15:23:31.641 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 7/7 2023-11-02T15:23:31.642 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9. 1/7 2023-11-02T15:23:31.642 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarc 2/7 2023-11-02T15:23:31.642 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.n 3/7 2023-11-02T15:23:31.642 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-11-02T15:23:31.643 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-11-02T15:23:31.643 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-11-02T15:23:31.705 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:02 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:31.848 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:31.849 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2023-11-02T15:23:31.850 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:31.850 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2023-11-02T15:23:31.850 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:31.850 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 7.4 M 2023-11-02T15:23:31.851 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:31.851 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-02T15:23:31.851 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-02T15:23:31.851 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-02T15:23:31.851 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-02T15:23:31.852 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-02T15:23:31.852 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-02T15:23:31.852 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-02T15:23:31.852 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-02T15:23:31.852 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================================ 2023-11-02T15:23:31.853 INFO:teuthology.orchestra.run.smithi175.stdout:Install 12 Packages 2023-11-02T15:23:31.854 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:31.854 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 37 M 2023-11-02T15:23:31.855 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 209 M 2023-11-02T15:23:31.855 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:31.959 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-11-02T15:23:31.959 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:31.960 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:31.960 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:31.960 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:31.960 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-02T15:23:31.961 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-02T15:23:31.961 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-02T15:23:31.961 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-02T15:23:31.961 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:31.962 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:31.977 INFO:teuthology.orchestra.run.smithi175.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 254 kB/s | 32 kB 00:00 2023-11-02T15:23:32.011 INFO:teuthology.orchestra.run.smithi175.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 532 kB/s | 17 kB 00:00 2023-11-02T15:23:32.044 INFO:teuthology.orchestra.run.smithi175.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-11-02T15:23:32.094 INFO:teuthology.orchestra.run.smithi175.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2023-11-02T15:23:32.155 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-diskprediction-local 2023-11-02T15:23:32.161 INFO:teuthology.orchestra.run.smithi175.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 3.1 MB/s | 205 kB 00:00 2023-11-02T15:23:32.219 INFO:teuthology.orchestra.run.smithi175.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 27 MB/s | 4.7 MB 00:00 2023-11-02T15:23:32.303 INFO:teuthology.orchestra.run.smithi175.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2023-11-02T15:23:32.328 INFO:teuthology.orchestra.run.smithi175.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2023-11-02T15:23:32.529 INFO:teuthology.orchestra.run.smithi175.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 70 MB/s | 16 MB 00:00 2023-11-02T15:23:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:(10/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2023-11-02T15:23:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2023-11-02T15:23:32.697 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:04 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:32.813 INFO:teuthology.orchestra.run.smithi175.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.0-1 7.7 MB/s | 7.4 MB 00:00 2023-11-02T15:23:32.815 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:32.816 INFO:teuthology.orchestra.run.smithi175.stdout:Total 38 MB/s | 37 MB 00:00 2023-11-02T15:23:32.816 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:32.838 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:32.838 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:32.841 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:32.842 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2023-11-02T15:23:32.843 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:32.843 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2023-11-02T15:23:32.843 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:32.843 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 7.4 M 2023-11-02T15:23:32.844 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:32.844 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-02T15:23:32.844 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-02T15:23:32.845 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-02T15:23:32.845 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-02T15:23:32.845 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-02T15:23:32.845 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-02T15:23:32.846 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-02T15:23:32.846 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-02T15:23:32.847 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-02T15:23:32.847 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-02T15:23:32.847 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-02T15:23:32.848 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:32.848 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:32.848 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2023-11-02T15:23:32.849 INFO:teuthology.orchestra.run.smithi174.stdout:Install 12 Packages 2023-11-02T15:23:32.849 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:32.850 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 37 M 2023-11-02T15:23:32.851 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 209 M 2023-11-02T15:23:32.851 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:32.952 INFO:teuthology.orchestra.run.smithi174.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 299 kB/s | 32 kB 00:00 2023-11-02T15:23:33.009 INFO:teuthology.orchestra.run.smithi174.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 491 kB/s | 17 kB 00:00 2023-11-02T15:23:33.043 INFO:teuthology.orchestra.run.smithi174.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-11-02T15:23:33.093 INFO:teuthology.orchestra.run.smithi174.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2023-11-02T15:23:33.152 INFO:teuthology.orchestra.run.smithi174.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 205 kB 00:00 2023-11-02T15:23:33.227 INFO:teuthology.orchestra.run.smithi174.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-11-02T15:23:33.253 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:33.253 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:33.302 INFO:teuthology.orchestra.run.smithi174.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2023-11-02T15:23:33.328 INFO:teuthology.orchestra.run.smithi174.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2023-11-02T15:23:33.528 INFO:teuthology.orchestra.run.smithi174.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 70 MB/s | 16 MB 00:00 2023-11-02T15:23:33.562 INFO:teuthology.orchestra.run.smithi174.stdout:(10/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2023-11-02T15:23:33.587 INFO:teuthology.orchestra.run.smithi174.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-11-02T15:23:33.970 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:34.044 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-11-02T15:23:34.084 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-11-02T15:23:34.178 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-11-02T15:23:34.212 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-11-02T15:23:34.391 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-11-02T15:23:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-11-02T15:23:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-11-02T15:23:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-11-02T15:23:35.212 INFO:teuthology.orchestra.run.smithi174.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.0-1 3.1 MB/s | 7.4 MB 00:02 2023-11-02T15:23:35.215 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:35.216 INFO:teuthology.orchestra.run.smithi174.stdout:Total 16 MB/s | 37 MB 00:02 2023-11-02T15:23:35.216 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:35.241 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:35.241 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:35.281 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-11-02T15:23:35.681 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:35.681 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:36.400 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:36.495 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-11-02T15:23:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-11-02T15:23:36.524 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-11-02T15:23:36.618 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-11-02T15:23:36.669 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-11-02T15:23:36.857 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-11-02T15:23:36.901 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-11-02T15:23:36.986 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-11-02T15:23:37.552 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-11-02T15:23:37.584 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 12/12 2023-11-02T15:23:37.652 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-11-02T15:23:37.763 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-11-02T15:23:38.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-11-02T15:23:40.017 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-11-02T15:23:40.050 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 12/12 2023-11-02T15:23:40.126 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 12/12 2023-11-02T15:23:40.126 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 1/12 2023-11-02T15:23:40.126 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-11-02T15:23:40.126 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-11-02T15:23:40.127 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-11-02T15:23:40.127 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-11-02T15:23:40.127 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-11-02T15:23:40.127 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-11-02T15:23:40.127 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-11-02T15:23:40.128 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-11-02T15:23:40.128 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-11-02T15:23:40.128 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-11-02T15:23:40.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-11-02T15:23:40.553 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:40.554 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:40.554 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:40.554 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-02T15:23:40.554 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-02T15:23:40.554 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-02T15:23:40.555 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-02T15:23:40.555 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-02T15:23:40.555 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-02T15:23:40.555 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-02T15:23:40.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-02T15:23:40.556 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-02T15:23:40.556 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-02T15:23:40.556 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-02T15:23:40.556 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:40.556 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:40.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-rook 2023-11-02T15:23:41.444 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:12 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:41.587 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:41.587 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:41.588 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:41.588 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:41.588 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:41.589 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 47 k 2023-11-02T15:23:41.589 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:41.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-02T15:23:41.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-02T15:23:41.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-02T15:23:41.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-02T15:23:41.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-02T15:23:41.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-02T15:23:41.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-02T15:23:41.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-02T15:23:41.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-02T15:23:41.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-02T15:23:41.592 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-02T15:23:41.592 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-02T15:23:41.592 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:41.593 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:41.593 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:41.593 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2023-11-02T15:23:41.594 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:41.595 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.0 M 2023-11-02T15:23:41.595 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 26 M 2023-11-02T15:23:41.595 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:41.792 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 204 kB/s | 24 kB 00:00 2023-11-02T15:23:41.817 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 125 kB/s | 18 kB 00:00 2023-11-02T15:23:41.859 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2023-11-02T15:23:41.884 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2023-11-02T15:23:41.901 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.0 MB/s | 211 kB 00:00 2023-11-02T15:23:41.926 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2023-11-02T15:23:42.018 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): ceph-mgr-rook-18.2.0-1159.gb5a9486b.el9 137 kB/s | 47 kB 00:00 2023-11-02T15:23:42.060 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2023-11-02T15:23:42.110 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 154 kB/s | 32 kB 00:00 2023-11-02T15:23:42.169 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): python3-google-auth-2.23.3-1.el9.noarc 1.3 MB/s | 197 kB 00:00 2023-11-02T15:23:42.203 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2023-11-02T15:23:42.228 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2023-11-02T15:23:42.253 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2023-11-02T15:23:42.257 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:42.257 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.1 MB/s | 2.0 MB 00:00 2023-11-02T15:23:42.257 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:42.269 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:42.270 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:42.365 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:42.365 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:42.603 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 12/12 2023-11-02T15:23:42.603 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a94 1/12 2023-11-02T15:23:42.603 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-11-02T15:23:42.603 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-11-02T15:23:42.604 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-11-02T15:23:42.604 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-11-02T15:23:42.604 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-11-02T15:23:42.604 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-11-02T15:23:42.604 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-11-02T15:23:42.605 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-11-02T15:23:42.605 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-11-02T15:23:42.605 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-11-02T15:23:42.881 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:43.051 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-11-02T15:23:43.062 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-11-02T15:23:43.062 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:43.063 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:43.063 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:43.063 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-02T15:23:43.063 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-02T15:23:43.063 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-02T15:23:43.064 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-02T15:23:43.064 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-02T15:23:43.064 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-02T15:23:43.064 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-02T15:23:43.064 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-02T15:23:43.065 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-02T15:23:43.065 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-02T15:23:43.065 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-02T15:23:43.065 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:43.065 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:43.103 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-11-02T15:23:43.168 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-11-02T15:23:43.205 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-11-02T15:23:43.247 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-11-02T15:23:43.339 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-11-02T15:23:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 7/13 2023-11-02T15:23:43.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-rook 2023-11-02T15:23:43.497 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-11-02T15:23:44.009 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:16 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:44.152 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:44.153 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:44.153 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:44.153 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:44.153 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:44.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 47 k 2023-11-02T15:23:44.154 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:44.154 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-02T15:23:44.154 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-02T15:23:44.154 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-02T15:23:44.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-02T15:23:44.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-02T15:23:44.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-02T15:23:44.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-02T15:23:44.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-02T15:23:44.156 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-02T15:23:44.156 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-02T15:23:44.156 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-02T15:23:44.156 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-02T15:23:44.156 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:44.157 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:44.157 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:44.157 INFO:teuthology.orchestra.run.smithi174.stdout:Install 13 Packages 2023-11-02T15:23:44.157 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:44.158 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.0 M 2023-11-02T15:23:44.158 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 26 M 2023-11-02T15:23:44.158 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:44.217 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-11-02T15:23:44.269 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-11-02T15:23:44.303 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-11-02T15:23:44.349 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-11-02T15:23:44.373 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 13/13 2023-11-02T15:23:44.617 INFO:teuthology.orchestra.run.smithi174.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 235 kB/s | 24 kB 00:00 2023-11-02T15:23:44.658 INFO:teuthology.orchestra.run.smithi174.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 124 kB/s | 18 kB 00:00 2023-11-02T15:23:44.700 INFO:teuthology.orchestra.run.smithi174.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.8 MB/s | 174 kB 00:00 2023-11-02T15:23:44.725 INFO:teuthology.orchestra.run.smithi174.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2023-11-02T15:23:44.751 INFO:teuthology.orchestra.run.smithi174.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2023-11-02T15:23:44.767 INFO:teuthology.orchestra.run.smithi174.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2023-11-02T15:23:44.843 INFO:teuthology.orchestra.run.smithi174.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 351 kB/s | 32 kB 00:00 2023-11-02T15:23:44.860 INFO:teuthology.orchestra.run.smithi174.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 163 kB/s | 15 kB 00:00 2023-11-02T15:23:44.919 INFO:teuthology.orchestra.run.smithi174.stdout:(9/13): python3-google-auth-2.23.3-1.el9.noarch 2.6 MB/s | 197 kB 00:00 2023-11-02T15:23:44.961 INFO:teuthology.orchestra.run.smithi174.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2023-11-02T15:23:45.002 INFO:teuthology.orchestra.run.smithi174.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2023-11-02T15:23:45.036 INFO:teuthology.orchestra.run.smithi174.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2023-11-02T15:23:45.061 INFO:teuthology.orchestra.run.smithi174.stdout:(13/13): ceph-mgr-rook-18.2.0-1159.gb5a9486b.el 87 kB/s | 47 kB 00:00 2023-11-02T15:23:45.063 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:45.064 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2023-11-02T15:23:45.064 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:45.076 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:45.076 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 13/13 2023-11-02T15:23:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 1/13 2023-11-02T15:23:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-11-02T15:23:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-11-02T15:23:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-11-02T15:23:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-11-02T15:23:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-11-02T15:23:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-11-02T15:23:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-11-02T15:23:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-11-02T15:23:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 10/13 2023-11-02T15:23:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-11-02T15:23:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-11-02T15:23:45.172 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:45.172 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:45.528 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-11-02T15:23:45.528 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:45.528 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:45.528 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:45.529 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-02T15:23:45.529 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-02T15:23:45.529 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-02T15:23:45.529 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-02T15:23:45.530 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-02T15:23:45.530 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-02T15:23:45.530 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-02T15:23:45.530 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-02T15:23:45.530 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-02T15:23:45.531 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-02T15:23:45.531 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-02T15:23:45.531 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-02T15:23:45.531 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:45.531 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:45.696 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:45.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-cephadm 2023-11-02T15:23:45.867 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-11-02T15:23:45.929 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-11-02T15:23:45.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-11-02T15:23:46.021 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-11-02T15:23:46.063 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-11-02T15:23:46.164 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-11-02T15:23:46.280 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 7/13 2023-11-02T15:23:46.325 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:17 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:46.330 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-11-02T15:23:46.478 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:46.478 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:46.479 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:46.479 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:46.479 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:46.479 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 132 k 2023-11-02T15:23:46.480 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:46.480 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-02T15:23:46.480 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-02T15:23:46.480 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-02T15:23:46.480 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-02T15:23:46.481 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-02T15:23:46.481 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:46.481 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:46.481 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:23:46.481 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2023-11-02T15:23:46.482 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:46.482 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 6.8 M 2023-11-02T15:23:46.482 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 33 M 2023-11-02T15:23:46.483 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:46.687 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.5 MB/s | 228 kB 00:00 2023-11-02T15:23:46.854 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2023-11-02T15:23:47.047 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-11-02T15:23:47.102 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-11-02T15:23:47.136 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-11-02T15:23:47.190 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-11-02T15:23:47.222 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 13/13 2023-11-02T15:23:47.231 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): ceph-mgr-cephadm-18.2.0-1159.gb5a9486b.e 190 kB/s | 132 kB 00:00 2023-11-02T15:23:48.096 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 13/13 2023-11-02T15:23:48.097 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 1/13 2023-11-02T15:23:48.097 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-11-02T15:23:48.097 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-11-02T15:23:48.097 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-11-02T15:23:48.097 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-11-02T15:23:48.098 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-11-02T15:23:48.098 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-11-02T15:23:48.098 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-11-02T15:23:48.098 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-11-02T15:23:48.099 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 10/13 2023-11-02T15:23:48.099 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-11-02T15:23:48.099 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-11-02T15:23:48.458 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 36 kB/s | 58 kB 00:01 2023-11-02T15:23:48.459 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-11-02T15:23:48.460 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:48.460 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:48.460 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:48.460 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-02T15:23:48.461 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-02T15:23:48.461 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-02T15:23:48.461 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-02T15:23:48.461 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-02T15:23:48.461 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-02T15:23:48.462 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-02T15:23:48.462 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-02T15:23:48.462 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-02T15:23:48.462 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-02T15:23:48.462 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-02T15:23:48.463 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-02T15:23:48.463 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:48.463 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:48.659 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-cephadm 2023-11-02T15:23:48.665 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 36 kB/s | 51 kB 00:01 2023-11-02T15:23:48.749 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 266 kB/s | 548 kB 00:02 2023-11-02T15:23:48.751 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:48.752 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.0 MB/s | 6.8 MB 00:02 2023-11-02T15:23:48.752 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:48.763 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:48.763 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:48.855 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:48.856 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:49.215 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:21 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:49.314 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-11-02T15:23:49.356 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:49.357 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:49.357 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:49.357 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:49.359 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:49.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 132 k 2023-11-02T15:23:49.359 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:49.359 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-02T15:23:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-02T15:23:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-02T15:23:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-02T15:23:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-02T15:23:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:49.361 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:49.361 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:23:49.361 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2023-11-02T15:23:49.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:49.362 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 6.8 M 2023-11-02T15:23:49.362 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 33 M 2023-11-02T15:23:49.362 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:49.367 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-11-02T15:23:49.574 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.4 MB/s | 228 kB 00:00 2023-11-02T15:23:49.662 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2023-11-02T15:23:49.885 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-11-02T15:23:49.904 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 264 kB/s | 58 kB 00:00 2023-11-02T15:23:49.929 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): ceph-mgr-cephadm-18.2.0-1159.gb5a9486b.e 254 kB/s | 132 kB 00:00 2023-11-02T15:23:49.963 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 872 kB/s | 51 kB 00:00 2023-11-02T15:23:49.969 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-11-02T15:23:50.005 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2023-11-02T15:23:50.009 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:50.009 INFO:teuthology.orchestra.run.smithi174.stdout:Total 10 MB/s | 6.8 MB 00:00 2023-11-02T15:23:50.009 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:50.018 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:50.019 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-11-02T15:23:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 6/6 2023-11-02T15:23:50.106 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:50.107 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:50.412 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:50.587 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-11-02T15:23:50.648 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-11-02T15:23:50.928 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 6/6 2023-11-02T15:23:50.928 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/6 2023-11-02T15:23:50.929 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-11-02T15:23:50.929 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-11-02T15:23:50.929 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-11-02T15:23:50.929 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-11-02T15:23:51.161 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-11-02T15:23:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-11-02T15:23:51.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:51.227 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:51.227 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-02T15:23:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-02T15:23:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-02T15:23:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-02T15:23:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-02T15:23:51.229 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:51.229 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:51.241 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-11-02T15:23:51.303 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-11-02T15:23:51.327 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 6/6 2023-11-02T15:23:51.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse 2023-11-02T15:23:52.012 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:23 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:52.153 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:52.153 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:52.153 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:52.154 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:52.154 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:52.154 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 840 k 2023-11-02T15:23:52.154 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:23:52.154 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-02T15:23:52.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:52.155 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:52.155 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:52.155 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2023-11-02T15:23:52.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:52.156 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 919 k 2023-11-02T15:23:52.156 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.7 M 2023-11-02T15:23:52.156 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:52.203 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 6/6 2023-11-02T15:23:52.203 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 1/6 2023-11-02T15:23:52.203 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-11-02T15:23:52.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-11-02T15:23:52.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-11-02T15:23:52.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-11-02T15:23:52.258 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 759 kB/s | 79 kB 00:00 2023-11-02T15:23:52.531 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-11-02T15:23:52.531 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:52.531 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:52.532 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:52.532 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-02T15:23:52.532 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-02T15:23:52.532 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-02T15:23:52.533 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-02T15:23:52.533 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-02T15:23:52.533 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:52.533 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:52.793 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-fuse 2023-11-02T15:23:53.017 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): ceph-fuse-18.2.0-1159.gb5a9486b.el9.x86_ 975 kB/s | 840 kB 00:00 2023-11-02T15:23:53.018 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:53.019 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.0 MB/s | 919 kB 00:00 2023-11-02T15:23:53.019 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:53.041 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:53.041 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:53.117 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:53.118 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:53.302 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:53.336 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:25 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-11-02T15:23:53.477 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:53.478 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:53.479 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:53.479 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 840 k 2023-11-02T15:23:53.479 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout:Install 2 Packages 2023-11-02T15:23:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:53.481 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 919 k 2023-11-02T15:23:53.481 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.7 M 2023-11-02T15:23:53.481 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:53.500 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:23:53.581 INFO:teuthology.orchestra.run.smithi174.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 771 kB/s | 79 kB 00:00 2023-11-02T15:23:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:23:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/2 2023-11-02T15:23:54.357 INFO:teuthology.orchestra.run.smithi174.stdout:(2/2): ceph-fuse-18.2.0-1159.gb5a9486b.el9.x86_ 958 kB/s | 840 kB 00:00 2023-11-02T15:23:54.358 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:54.358 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.0 MB/s | 919 kB 00:00 2023-11-02T15:23:54.359 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:54.379 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:54.379 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:54.455 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:54.455 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:54.474 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-11-02T15:23:54.474 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:54.475 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-11-02T15:23:54.475 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:54.653 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:54.758 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-volume 2023-11-02T15:23:54.795 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-11-02T15:23:54.863 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:23:55.316 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:26 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:55.463 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:55.464 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:55.464 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-11-02T15:23:55.465 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:55.465 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:55.465 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 262 k 2023-11-02T15:23:55.465 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:55.465 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:55.466 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:55.466 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:23:55.466 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:55.467 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 262 k 2023-11-02T15:23:55.467 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.2 M 2023-11-02T15:23:55.467 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:55.532 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:23:55.532 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/2 2023-11-02T15:23:55.866 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-11-02T15:23:55.866 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:55.867 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:55.867 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-11-02T15:23:55.867 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:55.867 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:56.145 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-volume 2023-11-02T15:23:56.271 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-volume-18.2.0-1159.gb5a9486b.el9.noarch.rp 325 kB/s | 262 kB 00:00 2023-11-02T15:23:56.272 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:56.272 INFO:teuthology.orchestra.run.smithi175.stdout:Total 324 kB/s | 262 kB 00:00 2023-11-02T15:23:56.272 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:56.280 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:56.280 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:56.313 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:56.313 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:56.527 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:56.573 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:56.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:56.588 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:56.588 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-02T15:23:56.589 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:56.705 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:28 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:56.848 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:23:56.849 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:56.849 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-02T15:23:56.850 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:56.850 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:23:56.850 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:18.2.0-1159.gb5a9486b.el9 ceph-noarch 262 k 2023-11-02T15:23:56.850 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:56.850 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:23:56.851 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:23:56.851 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:23:56.851 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:56.851 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 262 k 2023-11-02T15:23:56.852 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.2 M 2023-11-02T15:23:56.852 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:23:57.569 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-volume-18.2.0-1159.gb5a9486b.el9.noarch.rp 365 kB/s | 262 kB 00:00 2023-11-02T15:23:57.570 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:57.570 INFO:teuthology.orchestra.run.smithi174.stdout:Total 364 kB/s | 262 kB 00:00 2023-11-02T15:23:57.570 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:23:57.578 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:23:57.578 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:23:57.607 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:57.608 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:57.608 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:23:57.608 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:57.608 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:57.608 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:23:57.612 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:23:57.613 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:23:57.824 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:23:57.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:57.911 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:57.912 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:23:57.912 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-02T15:23:57.912 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:57.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel 2023-11-02T15:23:58.490 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:29 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:23:58.634 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:23:58.635 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:23:58.636 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:58.636 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:23:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 127 k 2023-11-02T15:23:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:58.636 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 127 k 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 456 k 2023-11-02T15:23:58.637 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:23:58.994 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 1/1 2023-11-02T15:23:58.995 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:58.995 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:23:58.995 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:18.2.0-1159.gb5a9486b.el9.noarch 2023-11-02T15:23:58.995 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:23:58.995 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:23:59.169 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-18.2.0-1159.gb5a9486b.el9.x86_64 238 kB/s | 127 kB 00:00 2023-11-02T15:23:59.169 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:23:59.170 INFO:teuthology.orchestra.run.smithi175.stdout:Total 238 kB/s | 127 kB 00:00 2023-11-02T15:23:59.170 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:23:59.175 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:23:59.175 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:23:59.231 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:23:59.232 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:23:59.311 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados-devel 2023-11-02T15:23:59.379 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:23:59.475 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:23:59.877 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:31 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:23:59.946 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:00.022 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:00.023 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:00.023 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:24:00.023 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:00.024 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:00.024 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 127 k 2023-11-02T15:24:00.024 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:00.024 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:00.024 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:00.025 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:00.025 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:00.025 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 127 k 2023-11-02T15:24:00.025 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 456 k 2023-11-02T15:24:00.026 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:00.267 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:00.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:00.268 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:00.268 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:00.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:00.269 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:00.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2 2023-11-02T15:24:00.565 INFO:teuthology.orchestra.run.smithi174.stdout:librados-devel-18.2.0-1159.gb5a9486b.el9.x86_64 234 kB/s | 127 kB 00:00 2023-11-02T15:24:00.565 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:00.566 INFO:teuthology.orchestra.run.smithi174.stdout:Total 233 kB/s | 127 kB 00:00 2023-11-02T15:24:00.566 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:00.571 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:00.571 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:00.627 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:00.627 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:00.781 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:00.880 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:01.087 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:32 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:01.182 INFO:teuthology.orchestra.run.smithi175.stdout:Package libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:01.232 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:01.233 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:01.233 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:01.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs-devel 2023-11-02T15:24:01.304 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:01.617 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:01.617 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:01.617 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:01.618 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:01.618 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:01.618 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:01.854 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:32 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:01.884 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs2 2023-11-02T15:24:01.996 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:01.997 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:01.998 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:24:01.998 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:01.998 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:01.998 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 31 k 2023-11-02T15:24:01.999 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:01.999 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:01.999 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:01.999 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:24:01.999 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:02.000 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 31 k 2023-11-02T15:24:02.000 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 139 k 2023-11-02T15:24:02.000 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:02.443 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:34 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:02.479 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs-devel-18.2.0-1159.gb5a9486b.el9.x86_6 64 kB/s | 31 kB 00:00 2023-11-02T15:24:02.480 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:02.480 INFO:teuthology.orchestra.run.smithi175.stdout:Total 64 kB/s | 31 kB 00:00 2023-11-02T15:24:02.481 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:02.484 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:02.484 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:02.512 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:02.513 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:24:02.538 INFO:teuthology.orchestra.run.smithi174.stdout:Package libcephfs2-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:02.590 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:02.591 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:02.591 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:02.606 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:24:02.650 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs-devel 2023-11-02T15:24:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:03.207 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:35 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:03.350 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:03.351 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:24:03.351 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:03.351 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 31 k 2023-11-02T15:24:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:03.352 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:03.352 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:03.352 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:03.353 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 31 k 2023-11-02T15:24:03.353 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 139 k 2023-11-02T15:24:03.353 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:03.356 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:03.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:03.357 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:03.357 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:03.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:03.357 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:03.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2 2023-11-02T15:24:03.767 INFO:teuthology.orchestra.run.smithi174.stdout:libcephfs-devel-18.2.0-1159.gb5a9486b.el9.x86_6 75 kB/s | 31 kB 00:00 2023-11-02T15:24:03.768 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:03.768 INFO:teuthology.orchestra.run.smithi174.stdout:Total 74 kB/s | 31 kB 00:00 2023-11-02T15:24:03.769 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:03.772 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:03.772 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:03.799 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:03.799 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:03.880 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:03.999 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:04.108 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:35 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:04.199 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:04.248 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:04.249 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:04.249 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:04.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1 2023-11-02T15:24:04.389 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:04.655 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:04.656 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:04.809 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados2 2023-11-02T15:24:04.857 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:35 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:04.947 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:04.996 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:04.997 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:04.998 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:05.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados 2023-11-02T15:24:05.365 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:37 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:05.456 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:05.506 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:05.507 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:05.507 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:05.563 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librbd1 2023-11-02T15:24:05.607 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:36 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:05.699 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:05.750 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:05.751 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:05.751 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:05.819 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw 2023-11-02T15:24:06.110 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:38 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:06.203 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:06.253 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:06.254 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:06.254 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:06.317 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rados 2023-11-02T15:24:06.366 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:37 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:06.459 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:06.509 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:06.510 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:06.510 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:06.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs 2023-11-02T15:24:06.863 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:38 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:06.955 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rados-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:07.005 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:07.006 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:07.006 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:07.072 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rgw 2023-11-02T15:24:07.136 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:38 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:07.227 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:07.278 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:07.279 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:07.279 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:07.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd 2023-11-02T15:24:07.626 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:39 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:07.719 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rgw-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:07.768 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:07.769 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:07.770 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:07.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-cephfs 2023-11-02T15:24:07.890 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:38 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:07.982 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:08.031 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:08.032 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:24:08.033 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:08.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse 2023-11-02T15:24:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:40 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:08.483 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-cephfs-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:08.532 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:08.533 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:08.533 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:08.601 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rbd 2023-11-02T15:24:08.641 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:39 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:08.783 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:08.784 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:08.784 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-11-02T15:24:08.784 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:08.785 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:08.785 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 86 k 2023-11-02T15:24:08.785 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:08.785 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:08.785 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:08.786 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:24:08.786 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:08.786 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 86 k 2023-11-02T15:24:08.786 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 231 k 2023-11-02T15:24:08.786 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:09.150 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:41 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:09.243 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:24:09.293 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:09.294 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:24:09.294 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:09.359 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-fuse 2023-11-02T15:24:09.407 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 138 kB/s | 86 kB 00:00 2023-11-02T15:24:09.408 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:09.408 INFO:teuthology.orchestra.run.smithi175.stdout:Total 138 kB/s | 86 kB 00:00 2023-11-02T15:24:09.408 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:09.415 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:09.415 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:09.471 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:09.471 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:24:09.609 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:24:09.714 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:09.910 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:41 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:10.050 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:10.053 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:10.053 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2023-11-02T15:24:10.053 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:10.054 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:10.054 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 86 k 2023-11-02T15:24:10.054 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:10.054 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:10.055 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:10.055 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:10.055 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:10.055 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 86 k 2023-11-02T15:24:10.055 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 231 k 2023-11-02T15:24:10.056 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:10.123 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:10.441 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:10.441 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:10.441 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:10.442 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:10.442 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:10.442 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:10.604 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-fuse-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 156 kB/s | 86 kB 00:00 2023-11-02T15:24:10.605 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:10.605 INFO:teuthology.orchestra.run.smithi174.stdout:Total 155 kB/s | 86 kB 00:00 2023-11-02T15:24:10.605 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:10.612 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:10.612 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:10.666 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:10.666 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:10.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror 2023-11-02T15:24:10.804 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:10.902 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:11.232 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:42 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:11.299 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:11.380 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:11.381 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:11.382 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:24:11.382 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:11.382 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:11.382 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.0 M 2023-11-02T15:24:11.383 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:11.383 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:11.383 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:11.383 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:24:11.383 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:11.384 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.0 M 2023-11-02T15:24:11.384 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 12 M 2023-11-02T15:24:11.384 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:11.616 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:11.616 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:11.616 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:11.617 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:11.617 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:11.617 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:11.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-mirror 2023-11-02T15:24:12.393 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:44 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:12.518 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2023-11-02T15:24:12.519 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:12.519 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.6 MB/s | 3.0 MB 00:01 2023-11-02T15:24:12.519 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:12.529 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:12.529 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:12.540 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:12.540 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:12.541 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:24:12.541 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:12.541 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:12.541 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 3.0 M 2023-11-02T15:24:12.541 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:12.542 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:12.542 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:12.542 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:12.542 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:12.543 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 3.0 M 2023-11-02T15:24:12.543 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 12 M 2023-11-02T15:24:12.543 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:12.603 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:12.604 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:24:13.013 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:24:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:13.110 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:24:13.110 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-02T15:24:13.110 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-02T15:24:13.110 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-02T15:24:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:13.798 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-mirror-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2023-11-02T15:24:13.798 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:13.799 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.4 MB/s | 3.0 MB 00:01 2023-11-02T15:24:13.799 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:13.809 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:13.809 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:13.882 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:13.938 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:14.248 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:14.248 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:14.297 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:14.390 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-02T15:24:14.390 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-02T15:24:14.391 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-02T15:24:14.391 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-02T15:24:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:14.461 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd 2023-11-02T15:24:15.017 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:46 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:15.156 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:15.157 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:15.157 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-11-02T15:24:15.157 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:15.157 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:15.158 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 170 k 2023-11-02T15:24:15.158 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:15.158 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:15.158 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:15.158 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:24:15.159 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:15.159 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 170 k 2023-11-02T15:24:15.159 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 490 k 2023-11-02T15:24:15.159 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:15.562 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:15.562 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:15.562 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:15.563 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:15.563 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:15.563 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:15.621 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 366 kB/s | 170 kB 00:00 2023-11-02T15:24:15.622 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:15.622 INFO:teuthology.orchestra.run.smithi175.stdout:Total 365 kB/s | 170 kB 00:00 2023-11-02T15:24:15.622 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:15.630 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:15.630 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:15.685 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:15.685 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:24:15.774 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-nbd 2023-11-02T15:24:15.841 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:24:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:16.335 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:48 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:16.372 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:16.477 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:16.478 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:16.478 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2023-11-02T15:24:16.478 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:16.479 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:16.479 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 170 k 2023-11-02T15:24:16.479 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:16.479 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:16.479 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:16.480 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:16.480 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:16.480 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 170 k 2023-11-02T15:24:16.480 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 490 k 2023-11-02T15:24:16.480 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:16.700 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:16.701 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:16.701 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:16.701 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:16.701 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:16.701 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:16.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base-debuginfo 2023-11-02T15:24:17.034 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd-18.2.0-1159.gb5a9486b.el9.x86_64.rpm 306 kB/s | 170 kB 00:00 2023-11-02T15:24:17.035 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:17.035 INFO:teuthology.orchestra.run.smithi174.stdout:Total 305 kB/s | 170 kB 00:00 2023-11-02T15:24:17.035 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:17.043 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:17.043 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:17.097 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:17.098 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:17.247 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:17.357 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:17.496 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:48 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:17.643 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:17.644 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:17.644 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:24:17.645 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:17.645 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:17.645 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 74 M 2023-11-02T15:24:17.645 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-11-02T15:24:17.645 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 20 M 2023-11-02T15:24:17.646 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:17.646 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:17.646 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:17.646 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2023-11-02T15:24:17.646 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:17.647 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 94 M 2023-11-02T15:24:17.647 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 512 M 2023-11-02T15:24:17.647 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:17.873 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:18.146 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:24:18.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:18.147 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:18.147 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:18.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:18.148 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:18.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-base-debuginfo 2023-11-02T15:24:18.969 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:50 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:19.116 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:19.117 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:19.117 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:24:19.118 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:19.118 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:19.118 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 74 M 2023-11-02T15:24:19.118 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-02T15:24:19.118 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 20 M 2023-11-02T15:24:19.119 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:19.119 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:19.119 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:19.119 INFO:teuthology.orchestra.run.smithi174.stdout:Install 2 Packages 2023-11-02T15:24:19.119 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:19.120 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 94 M 2023-11-02T15:24:19.120 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 512 M 2023-11-02T15:24:19.120 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:20.007 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): ceph-debuginfo-18.2.0-1159.gb5a9486b.el9 8.7 MB/s | 20 MB 00:02 2023-11-02T15:24:22.612 INFO:teuthology.orchestra.run.smithi174.stdout:(1/2): ceph-debuginfo-18.2.0-1159.gb5a9486b.el9 5.9 MB/s | 20 MB 00:03 2023-11-02T15:24:23.090 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): ceph-base-debuginfo-18.2.0-1159.gb5a9486 14 MB/s | 74 MB 00:05 2023-11-02T15:24:23.092 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:23.093 INFO:teuthology.orchestra.run.smithi175.stdout:Total 17 MB/s | 94 MB 00:05 2023-11-02T15:24:23.093 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:23.097 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:23.098 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:23.710 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:23.711 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:24:26.624 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:24:33.811 INFO:teuthology.orchestra.run.smithi174.stdout:(2/2): ceph-base-debuginfo-18.2.0-1159.gb5a9486 5.0 MB/s | 74 MB 00:14 2023-11-02T15:24:33.813 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:33.814 INFO:teuthology.orchestra.run.smithi174.stdout:Total 6.4 MB/s | 94 MB 00:14 2023-11-02T15:24:33.814 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:24:33.818 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:24:33.819 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:24:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/2 2023-11-02T15:24:34.383 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 2/2 2023-11-02T15:24:34.432 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:24:34.433 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:24:37.346 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:24:37.793 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 2/2 2023-11-02T15:24:37.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/2 2023-11-02T15:24:38.068 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:24:38.068 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:38.068 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:24:38.068 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:38.069 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:38.069 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:38.069 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:24:38.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-common-debuginfo 2023-11-02T15:24:38.825 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:09 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:24:38.967 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:24:38.968 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:38.968 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:24:38.968 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:38.968 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:24:38.969 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 404 M 2023-11-02T15:24:38.969 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:38.969 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:24:38.969 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:24:38.969 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:24:38.970 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:24:38.970 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 404 M 2023-11-02T15:24:38.970 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.7 G 2023-11-02T15:24:38.970 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:24:45.008 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/2 2023-11-02T15:24:45.152 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 2/2 2023-11-02T15:24:48.427 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 2/2 2023-11-02T15:24:48.428 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/2 2023-11-02T15:24:48.706 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2/2 2023-11-02T15:24:48.706 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:48.706 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:24:48.707 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:48.707 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:24:48.707 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:48.707 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:24:48.944 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-common-debuginfo 2023-11-02T15:24:49.520 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:02:21 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:24:49.670 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:24:49.671 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:24:49.671 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:49.671 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:24:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 404 M 2023-11-02T15:24:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:49.672 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:24:49.672 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:24:49.673 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:24:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:24:49.673 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 404 M 2023-11-02T15:24:49.673 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.7 G 2023-11-02T15:24:49.674 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:24:57.300 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-common-debuginfo-18.2.0-1159.gb5a9486b.el9 22 MB/s | 404 MB 00:18 2023-11-02T15:24:57.301 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:24:57.301 INFO:teuthology.orchestra.run.smithi175.stdout:Total 22 MB/s | 404 MB 00:18 2023-11-02T15:24:57.302 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:24:57.306 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:24:57.306 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:24:59.732 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:24:59.733 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:25:17.771 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-common-debuginfo-18.2.0-1159.gb5a9486b.el9 14 MB/s | 404 MB 00:28 2023-11-02T15:25:17.772 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:25:17.772 INFO:teuthology.orchestra.run.smithi174.stdout:Total 14 MB/s | 404 MB 00:28 2023-11-02T15:25:17.773 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:25:17.776 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:25:17.777 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:25:20.195 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:25:20.195 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:25:41.653 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:25:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:25:49.567 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:25:49.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:25:49.831 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:49.831 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:25:49.831 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:25:49.831 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:49.832 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:25:50.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-11-02T15:25:50.618 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:03:21 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:25:50.768 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:25:50.769 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:25:50.770 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:25:50.770 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:25:50.770 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:25:50.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.4 M 2023-11-02T15:25:50.770 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:50.771 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:25:50.771 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================== 2023-11-02T15:25:50.771 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:25:50.771 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:50.771 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 1.4 M 2023-11-02T15:25:50.772 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 6.2 M 2023-11-02T15:25:50.772 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:25:51.238 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-debuginfo-18.2.0-11 3.0 MB/s | 1.4 MB 00:00 2023-11-02T15:25:51.239 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:25:51.239 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.0 MB/s | 1.4 MB 00:00 2023-11-02T15:25:51.239 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:25:51.243 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:25:51.243 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:25:51.285 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:25:51.286 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:25:51.547 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:25:51.662 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:25:52.121 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:25:52.384 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:25:52.384 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:52.385 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:25:52.385 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:25:52.385 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:52.385 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:25:52.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw-debuginfo 2023-11-02T15:25:53.100 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:03:24 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:25:53.242 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:25:53.243 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:25:53.243 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:25:53.243 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:25:53.244 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:25:53.244 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 163 M 2023-11-02T15:25:53.244 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:53.244 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:25:53.245 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:25:53.245 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:25:53.245 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:25:53.245 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 163 M 2023-11-02T15:25:53.245 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 680 M 2023-11-02T15:25:53.246 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:26:00.390 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-radosgw-debuginfo-18.2.0-1159.gb5a9486b.el 23 MB/s | 163 MB 00:07 2023-11-02T15:26:00.391 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:26:00.392 INFO:teuthology.orchestra.run.smithi175.stdout:Total 23 MB/s | 163 MB 00:07 2023-11-02T15:26:00.392 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:26:00.396 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:26:00.396 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:26:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:26:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:26:01.607 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:26:01.698 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:26:10.356 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:26:10.626 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:26:10.626 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:10.626 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:26:10.627 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:26:10.627 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:10.627 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:26:10.870 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-11-02T15:26:11.413 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:03:43 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:26:11.555 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:26:11.556 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:26:11.557 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:26:11.557 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:26:11.558 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:26:11.558 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.4 M 2023-11-02T15:26:11.558 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:11.559 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:26:11.559 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2023-11-02T15:26:11.559 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:26:11.560 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:11.560 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 1.4 M 2023-11-02T15:26:11.561 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 6.2 M 2023-11-02T15:26:11.561 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:26:11.860 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-immutable-object-cache-debuginfo-18.2.0-11 4.7 MB/s | 1.4 MB 00:00 2023-11-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi174.stdout:Total 4.6 MB/s | 1.4 MB 00:00 2023-11-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:26:11.905 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:26:11.905 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:26:12.151 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:26:12.241 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:26:12.630 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:26:12.902 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.2.0-1159. 1/1 2023-11-02T15:26:12.902 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:12.902 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:26:12.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:26:12.903 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:12.903 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:26:13.074 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw-debuginfo 2023-11-02T15:26:13.621 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:03:45 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:26:13.761 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:26:13.762 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:13.763 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:26:13.763 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:13.763 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:26:13.763 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 163 M 2023-11-02T15:26:13.763 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:13.764 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:26:13.764 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:13.764 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:26:13.764 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:13.764 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 163 M 2023-11-02T15:26:13.765 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 680 M 2023-11-02T15:26:13.765 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:26:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:26:18.178 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:22.579 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:22.579 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:26:22.580 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:26:22.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:26:22.580 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:26:22.580 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:26:22.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test-debuginfo 2023-11-02T15:26:23.305 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:03:54 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:26:23.446 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:26:23.447 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:26:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:26:23.448 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:26:23.448 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:26:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 602 M 2023-11-02T15:26:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:26:23.448 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:26:23.449 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:26:23.449 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:26:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:26:23.449 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 602 M 2023-11-02T15:26:23.449 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.7 G 2023-11-02T15:26:23.450 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:26:26.270 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-radosgw-debuginfo-18.2.0-1159.gb5a9486b.el 13 MB/s | 163 MB 00:12 2023-11-02T15:26:26.271 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:26:26.272 INFO:teuthology.orchestra.run.smithi174.stdout:Total 13 MB/s | 163 MB 00:12 2023-11-02T15:26:26.272 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:26:26.276 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:26:26.277 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:26:27.265 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:26:27.268 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:26:43.853 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:26:43.942 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:48.012 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:48.261 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x 1/1 2023-11-02T15:26:48.262 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:48.262 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:26:48.262 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:26:48.262 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:48.262 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:26:48.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-test-debuginfo 2023-11-02T15:26:48.993 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:04:20 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:26:49.134 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:26:49.135 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:49.135 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:26:49.136 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:49.136 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:26:49.136 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 602 M 2023-11-02T15:26:49.136 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 602 M 2023-11-02T15:26:49.137 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.7 G 2023-11-02T15:26:49.138 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:27:02.676 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-test-debuginfo-18.2.0-1159.gb5a9486b.el9.x 15 MB/s | 602 MB 00:39 2023-11-02T15:27:02.676 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:27:02.677 INFO:teuthology.orchestra.run.smithi175.stdout:Total 15 MB/s | 602 MB 00:39 2023-11-02T15:27:02.677 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:27:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:27:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:27:06.295 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:27:06.295 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:27:40.232 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-test-debuginfo-18.2.0-1159.gb5a9486b.el9.x 12 MB/s | 602 MB 00:51 2023-11-02T15:27:40.232 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:27:40.233 INFO:teuthology.orchestra.run.smithi174.stdout:Total 12 MB/s | 602 MB 00:51 2023-11-02T15:27:40.233 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:27:40.238 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:27:40.239 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:27:43.807 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:27:43.808 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:28:09.447 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:28:09.637 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:18.275 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:18.539 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:18.540 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:18.540 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:28:18.540 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:28:18.540 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:18.541 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:28:18.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base-debuginfo 2023-11-02T15:28:19.331 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:50 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:28:19.424 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:28:19.475 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:28:19.476 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-11-02T15:28:19.476 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:28:19.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-debuginfo 2023-11-02T15:28:20.088 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:51 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:28:20.230 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:28:20.231 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:28:20.231 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:20.232 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:28:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 19 M 2023-11-02T15:28:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:20.232 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:28:20.232 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:20.233 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:28:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:20.233 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 19 M 2023-11-02T15:28:20.234 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 101 M 2023-11-02T15:28:20.234 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:28:21.299 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-mgr-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 18 MB/s | 19 MB 00:01 2023-11-02T15:28:21.300 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:28:21.301 INFO:teuthology.orchestra.run.smithi175.stdout:Total 18 MB/s | 19 MB 00:01 2023-11-02T15:28:21.301 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:28:21.304 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:28:21.304 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:28:21.452 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:28:21.453 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:28:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:28:23.806 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:24.797 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:25.059 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:28:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:28:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:25.059 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:28:25.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mds-debuginfo 2023-11-02T15:28:25.761 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:05:56 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:28:25.904 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:28:25.905 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:25.905 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:28:25.906 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:25.906 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:28:25.906 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 48 M 2023-11-02T15:28:25.906 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:25.906 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:28:25.907 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:25.907 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:28:25.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:25.907 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 48 M 2023-11-02T15:28:25.908 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 203 M 2023-11-02T15:28:25.908 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:28:30.327 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-mds-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 11 MB/s | 48 MB 00:04 2023-11-02T15:28:30.328 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:28:30.328 INFO:teuthology.orchestra.run.smithi175.stdout:Total 11 MB/s | 48 MB 00:04 2023-11-02T15:28:30.328 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:28:30.332 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:28:30.332 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:28:30.654 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:28:30.655 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:28:35.665 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:28:35.765 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:37.588 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:28:37.588 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:37.589 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:28:37.589 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:28:37.589 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:37.589 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:28:37.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mon-debuginfo 2023-11-02T15:28:38.309 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:06:09 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:28:38.452 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:28:38.453 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:38.453 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:28:38.453 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:38.453 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:28:38.453 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 113 M 2023-11-02T15:28:38.454 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:38.454 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 113 M 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 525 M 2023-11-02T15:28:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:28:45.460 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-mon-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 16 MB/s | 113 MB 00:07 2023-11-02T15:28:45.461 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:28:45.462 INFO:teuthology.orchestra.run.smithi175.stdout:Total 16 MB/s | 113 MB 00:07 2023-11-02T15:28:45.462 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:28:45.466 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:28:45.466 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:28:46.169 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:28:46.169 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:28:46.778 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:28:46.879 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:55.273 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:55.559 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:28:55.559 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:28:55.559 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:28:55.560 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:28:55.560 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:28:55.560 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:28:55.813 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-base-debuginfo 2023-11-02T15:28:56.364 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:06:28 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:28:56.456 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-base-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 is already installed. 2023-11-02T15:28:56.507 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:28:56.508 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-02T15:28:56.508 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:28:56.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-debuginfo 2023-11-02T15:28:57.119 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:06:29 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:28:57.261 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:28:57.262 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:28:57.262 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:28:57.262 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:28:57.262 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:28:57.265 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 19 M 2023-11-02T15:28:57.265 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:28:57.265 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:28:57.266 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:28:57.266 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:28:57.266 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:28:57.267 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 19 M 2023-11-02T15:28:57.267 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 101 M 2023-11-02T15:28:57.267 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:28:58.113 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:28:58.298 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:00.134 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-mgr-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 6.7 MB/s | 19 MB 00:02 2023-11-02T15:29:00.135 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:29:00.135 INFO:teuthology.orchestra.run.smithi174.stdout:Total 6.7 MB/s | 19 MB 00:02 2023-11-02T15:29:00.136 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:29:00.138 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:29:00.139 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:29:00.286 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:29:00.286 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:29:01.620 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:01.908 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:01.908 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:29:01.909 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:29:01.909 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:29:01.909 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:29:01.909 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:29:02.066 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-osd-debuginfo 2023-11-02T15:29:02.458 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:29:02.564 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:02.616 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:06:33 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:29:02.757 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:29:02.757 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:29:02.758 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:29:02.758 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:29:02.758 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:29:02.758 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 312 M 2023-11-02T15:29:02.759 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:29:02.759 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:29:02.759 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:29:02.759 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:29:02.760 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:29:02.760 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 312 M 2023-11-02T15:29:02.760 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.3 G 2023-11-02T15:29:02.760 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:29:03.578 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:03.849 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:03.849 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:03.849 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:29:03.850 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:29:03.850 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:03.850 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:29:04.016 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mds-debuginfo 2023-11-02T15:29:04.564 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:06:36 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:29:04.704 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:29:04.705 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:04.706 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:29:04.706 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:04.706 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:29:04.706 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 48 M 2023-11-02T15:29:04.706 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:04.707 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:29:04.707 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:04.707 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:29:04.707 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:04.708 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 48 M 2023-11-02T15:29:04.708 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 203 M 2023-11-02T15:29:04.708 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:29:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-mds-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 4.1 MB/s | 48 MB 00:11 2023-11-02T15:29:16.644 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:29:16.644 INFO:teuthology.orchestra.run.smithi174.stdout:Total 4.1 MB/s | 48 MB 00:11 2023-11-02T15:29:16.645 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:29:16.648 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:29:16.649 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:29:16.968 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:29:16.969 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:29:21.974 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:29:22.075 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:22.541 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-osd-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 16 MB/s | 312 MB 00:19 2023-11-02T15:29:22.542 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:29:22.543 INFO:teuthology.orchestra.run.smithi175.stdout:Total 16 MB/s | 312 MB 00:19 2023-11-02T15:29:22.543 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:29:22.547 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:29:22.547 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:29:23.639 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:23.913 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:23.913 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:23.914 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:29:23.914 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:29:23.914 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:23.914 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:29:24.102 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mon-debuginfo 2023-11-02T15:29:24.428 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:29:24.429 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:29:24.653 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:06:56 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:29:24.794 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:29:24.795 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:24.796 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:29:24.796 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:24.796 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:29:24.796 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 113 M 2023-11-02T15:29:24.796 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 113 M 2023-11-02T15:29:24.797 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 525 M 2023-11-02T15:29:24.798 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:29:34.694 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-mon-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 11 MB/s | 113 MB 00:09 2023-11-02T15:29:34.695 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:29:34.696 INFO:teuthology.orchestra.run.smithi174.stdout:Total 11 MB/s | 113 MB 00:09 2023-11-02T15:29:34.696 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:29:34.699 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:29:34.700 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:29:35.396 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:29:35.397 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:29:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:29:47.499 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:50.811 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:51.092 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:29:51.092 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:51.092 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:29:51.092 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:29:51.093 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:51.093 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:29:51.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-osd-debuginfo 2023-11-02T15:29:51.833 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:07:23 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:29:51.972 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:29:51.973 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:51.973 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:29:51.973 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:51.973 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:29:51.974 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 312 M 2023-11-02T15:29:51.974 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:51.974 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:29:51.974 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:29:51.974 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:29:51.975 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:29:51.975 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 312 M 2023-11-02T15:29:51.975 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.3 G 2023-11-02T15:29:51.975 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:29:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:29:56.459 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:04.423 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:04.681 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:04.681 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:04.682 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:30:04.682 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:04.682 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:04.682 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:30:04.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse-debuginfo 2023-11-02T15:30:05.431 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:07:36 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:30:05.572 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:30:05.572 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:05.572 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:30:05.573 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:05.573 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:30:05.573 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 5.7 M 2023-11-02T15:30:05.573 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:05.573 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:30:05.574 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:05.574 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:30:05.574 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:05.574 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 5.7 M 2023-11-02T15:30:05.575 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 28 M 2023-11-02T15:30:05.575 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:30:06.055 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-fuse-debuginfo-18.2.0-1159.gb5a9486b.el9.x 12 MB/s | 5.7 MB 00:00 2023-11-02T15:30:06.056 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:06.056 INFO:teuthology.orchestra.run.smithi175.stdout:Total 12 MB/s | 5.7 MB 00:00 2023-11-02T15:30:06.057 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:30:06.060 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:30:06.060 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:30:06.126 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:30:06.127 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:30:06.786 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-osd-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 21 MB/s | 312 MB 00:14 2023-11-02T15:30:06.786 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:06.787 INFO:teuthology.orchestra.run.smithi174.stdout:Total 21 MB/s | 312 MB 00:14 2023-11-02T15:30:06.787 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:30:06.791 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:30:06.791 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:30:06.864 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:30:06.970 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:07.518 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:07.789 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:30:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:07.790 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:30:07.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel-debuginfo 2023-11-02T15:30:08.492 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:07:39 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:30:08.633 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:30:08.633 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:08.634 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:30:08.634 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:08.634 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:30:08.634 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 108 k 2023-11-02T15:30:08.634 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:08.635 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:30:08.635 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:08.635 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:30:08.635 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:08.636 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 108 k 2023-11-02T15:30:08.636 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 440 k 2023-11-02T15:30:08.636 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:30:08.664 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:30:09.499 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-debuginfo-18.2.0-1159.gb5a9486b. 125 kB/s | 108 kB 00:00 2023-11-02T15:30:09.500 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:09.501 INFO:teuthology.orchestra.run.smithi175.stdout:Total 125 kB/s | 108 kB 00:00 2023-11-02T15:30:09.501 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:30:09.504 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:30:09.504 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:30:09.539 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:30:09.540 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:30:09.652 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:30:09.746 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:30:10.135 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:30:10.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:30:10.402 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:10.402 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:30:10.402 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:10.403 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:10.403 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:30:10.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2-debuginfo 2023-11-02T15:30:11.119 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:07:42 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.9 M 2023-11-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:11.263 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:30:11.263 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:11.263 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 4.9 M 2023-11-02T15:30:11.263 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 23 M 2023-11-02T15:30:11.263 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:30:12.260 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs2-debuginfo-18.2.0-1159.gb5a9486b.el9. 4.9 MB/s | 4.9 MB 00:00 2023-11-02T15:30:12.261 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:12.261 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.9 MB/s | 4.9 MB 00:00 2023-11-02T15:30:12.262 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:30:12.265 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:30:12.265 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:30:12.330 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:30:12.330 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:30:12.949 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:30:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:30:13.561 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:30:13.835 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:30:13.836 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:13.836 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:30:13.836 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:13.836 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:13.836 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:30:14.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2-debuginfo 2023-11-02T15:30:14.558 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:07:45 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:30:14.698 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:30:14.699 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:14.699 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:30:14.700 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:14.700 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:30:14.700 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 44 M 2023-11-02T15:30:14.700 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:14.700 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 44 M 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 238 M 2023-11-02T15:30:14.701 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:30:40.686 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:30:40.837 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:41.133 INFO:teuthology.orchestra.run.smithi175.stdout:librados2-debuginfo-18.2.0-1159.gb5a9486b.el9.x 1.7 MB/s | 44 MB 00:26 2023-11-02T15:30:41.134 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:41.135 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.7 MB/s | 44 MB 00:26 2023-11-02T15:30:41.135 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:30:41.139 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:30:41.139 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:30:41.443 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:30:41.444 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:30:46.407 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:30:46.498 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:48.209 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:48.444 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:48.487 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:48.487 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:48.487 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:30:48.488 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:48.488 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:48.488 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:30:48.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1-debuginfo 2023-11-02T15:30:48.716 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:30:48.716 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:48.716 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:30:48.716 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:48.717 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:48.717 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:30:48.912 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-fuse-debuginfo 2023-11-02T15:30:49.232 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:20 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:30:49.373 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:30:49.374 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:49.374 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:30:49.375 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:49.375 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:30:49.375 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 51 M 2023-11-02T15:30:49.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:49.376 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:30:49.376 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:30:49.376 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:30:49.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:30:49.376 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 51 M 2023-11-02T15:30:49.377 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 292 M 2023-11-02T15:30:49.377 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:30:49.460 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:08:21 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:30:49.602 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:30:49.603 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:30:49.603 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:49.603 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:30:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 5.7 M 2023-11-02T15:30:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:49.604 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:30:49.604 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:49.604 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:30:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:49.605 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 5.7 M 2023-11-02T15:30:49.605 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 28 M 2023-11-02T15:30:49.605 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:30:51.843 INFO:teuthology.orchestra.run.smithi175.stdout:librbd1-debuginfo-18.2.0-1159.gb5a9486b.el9.x86 21 MB/s | 51 MB 00:02 2023-11-02T15:30:51.844 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:51.845 INFO:teuthology.orchestra.run.smithi175.stdout:Total 21 MB/s | 51 MB 00:02 2023-11-02T15:30:51.845 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:30:51.849 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:30:51.849 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:30:52.192 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:30:52.192 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:30:56.433 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-fuse-debuginfo-18.2.0-1159.gb5a9486b.el9.x 859 kB/s | 5.7 MB 00:06 2023-11-02T15:30:56.434 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:56.435 INFO:teuthology.orchestra.run.smithi174.stdout:Total 859 kB/s | 5.7 MB 00:06 2023-11-02T15:30:56.435 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:30:56.439 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:30:56.439 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:30:56.510 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:30:56.510 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:30:57.231 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:30:57.331 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:57.910 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:30:58.099 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:30:58.180 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:30:58.180 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:58.180 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:30:58.181 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:30:58.181 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:58.181 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:30:58.327 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados-devel-debuginfo 2023-11-02T15:30:58.874 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:08:30 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:30:59.016 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:30:59.017 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:59.018 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:30:59.018 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:59.018 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:30:59.018 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 108 k 2023-11-02T15:30:59.019 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:59.019 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:30:59.019 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:30:59.019 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:30:59.020 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:30:59.020 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 108 k 2023-11-02T15:30:59.020 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 440 k 2023-11-02T15:30:59.020 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:30:59.228 INFO:teuthology.orchestra.run.smithi174.stdout:librados-devel-debuginfo-18.2.0-1159.gb5a9486b. 517 kB/s | 108 kB 00:00 2023-11-02T15:30:59.228 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:30:59.229 INFO:teuthology.orchestra.run.smithi174.stdout:Total 514 kB/s | 108 kB 00:00 2023-11-02T15:30:59.229 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:30:59.232 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:30:59.232 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:30:59.266 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:30:59.266 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:30:59.381 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:30:59.484 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:30:59.859 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:00.124 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:00.124 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:00.124 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:00.124 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:00.125 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:00.125 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:00.303 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs2-debuginfo 2023-11-02T15:31:00.445 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:00.445 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:00.445 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:00.445 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:00.446 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:00.446 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:00.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs-debuginfo 2023-11-02T15:31:00.850 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:08:32 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:00.990 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:00.991 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:00.991 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:00.992 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:00.992 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:00.992 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 4.9 M 2023-11-02T15:31:00.992 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:00.992 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:00.993 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:00.993 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:00.993 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:00.993 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 4.9 M 2023-11-02T15:31:00.993 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 23 M 2023-11-02T15:31:00.994 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:01.184 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:32 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:01.325 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:01.326 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:01.326 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:01.327 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 615 k 2023-11-02T15:31:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:01.327 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:01.327 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:01.328 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:01.328 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 615 k 2023-11-02T15:31:01.329 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.1 M 2023-11-02T15:31:01.329 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:01.704 INFO:teuthology.orchestra.run.smithi175.stdout:python3-cephfs-debuginfo-18.2.0-1159.gb5a9486b. 1.6 MB/s | 615 kB 00:00 2023-11-02T15:31:01.705 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:01.705 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.6 MB/s | 615 kB 00:00 2023-11-02T15:31:01.705 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:01.708 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:01.708 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:01.911 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:02.012 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:02.721 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:02.721 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:02.722 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:02.722 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:02.722 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:02.722 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:02.912 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados-debuginfo 2023-11-02T15:31:03.105 INFO:teuthology.orchestra.run.smithi174.stdout:libcephfs2-debuginfo-18.2.0-1159.gb5a9486b.el9. 2.3 MB/s | 4.9 MB 00:02 2023-11-02T15:31:03.106 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.3 MB/s | 4.9 MB 00:02 2023-11-02T15:31:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:03.111 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:03.111 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:03.175 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:03.176 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:03.461 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:34 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:03.603 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:03.604 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:03.605 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:03.605 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:03.605 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:03.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 770 k 2023-11-02T15:31:03.605 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:03.606 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:03.606 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:03.606 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:03.606 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:03.606 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 770 k 2023-11-02T15:31:03.607 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.7 M 2023-11-02T15:31:03.607 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:03.796 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:03.907 INFO:teuthology.orchestra.run.smithi175.stdout:python3-rados-debuginfo-18.2.0-1159.gb5a9486b.e 2.5 MB/s | 770 kB 00:00 2023-11-02T15:31:03.907 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:03.908 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.5 MB/s | 770 kB 00:00 2023-11-02T15:31:03.908 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:03.911 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:03.911 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:03.936 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:04.136 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:04.240 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:04.454 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:04.648 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:04.727 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:04.728 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:04.728 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:04.728 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:04.728 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:04.728 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:04.902 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados2-debuginfo 2023-11-02T15:31:04.909 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:04.910 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:04.910 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:04.910 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:04.910 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:04.910 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:05.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd-debuginfo 2023-11-02T15:31:05.453 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:08:37 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:05.594 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:05.595 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:05.595 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:05.595 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:05.596 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:05.596 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 44 M 2023-11-02T15:31:05.596 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:05.596 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:05.597 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:05.597 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:05.597 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:05.597 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 44 M 2023-11-02T15:31:05.597 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 238 M 2023-11-02T15:31:05.598 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:36 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:05.774 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:05.775 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:05.775 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.1 M 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:05.776 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:05.777 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:05.777 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:05.777 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 1.1 M 2023-11-02T15:31:05.777 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 4.3 M 2023-11-02T15:31:05.777 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:06.142 INFO:teuthology.orchestra.run.smithi175.stdout:python3-rbd-debuginfo-18.2.0-1159.gb5a9486b.el9 3.1 MB/s | 1.1 MB 00:00 2023-11-02T15:31:06.143 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:06.144 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.1 MB/s | 1.1 MB 00:00 2023-11-02T15:31:06.144 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:06.147 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:06.147 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:06.185 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:06.186 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:06.404 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:06.499 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:06.878 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:07.141 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:07.141 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:07.141 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:07.142 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:07.142 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:07.142 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:07.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw-debuginfo 2023-11-02T15:31:07.892 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:38 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:07.966 INFO:teuthology.orchestra.run.smithi174.stdout:librados2-debuginfo-18.2.0-1159.gb5a9486b.el9.x 19 MB/s | 44 MB 00:02 2023-11-02T15:31:07.967 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:07.968 INFO:teuthology.orchestra.run.smithi174.stdout:Total 19 MB/s | 44 MB 00:02 2023-11-02T15:31:07.968 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:07.972 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:07.972 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:08.032 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:08.033 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:08.033 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:08.034 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:08.034 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:08.034 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 338 k 2023-11-02T15:31:08.034 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:08.035 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:08.035 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:08.035 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:08.035 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:08.035 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 338 k 2023-11-02T15:31:08.036 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.0 M 2023-11-02T15:31:08.036 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:08.275 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:08.276 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:08.529 INFO:teuthology.orchestra.run.smithi175.stdout:python3-rgw-debuginfo-18.2.0-1159.gb5a9486b.el9 683 kB/s | 338 kB 00:00 2023-11-02T15:31:08.530 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:08.531 INFO:teuthology.orchestra.run.smithi175.stdout:Total 681 kB/s | 338 kB 00:00 2023-11-02T15:31:08.531 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:08.533 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:08.533 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:08.567 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:08.568 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:08.818 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:09.225 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:09.484 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:09.485 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:09.485 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:09.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:09.485 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:09.485 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:09.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse-debuginfo 2023-11-02T15:31:10.206 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:41 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:10.346 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:10.347 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:10.347 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:10.348 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:10.348 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:10.348 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 507 k 2023-11-02T15:31:10.348 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:10.348 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:10.349 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:10.349 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:10.349 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:10.349 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 507 k 2023-11-02T15:31:10.349 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.5 M 2023-11-02T15:31:10.350 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 579 kB/s | 507 kB 00:00 2023-11-02T15:31:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:Total 578 kB/s | 507 kB 00:00 2023-11-02T15:31:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:11.267 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:11.267 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:11.430 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:11.547 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:11.932 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:12.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:12.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:12.192 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:12.192 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:12.192 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:12.192 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:12.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror-debuginfo 2023-11-02T15:31:12.929 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:08:43 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:13.071 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:13.072 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:13.072 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:13.072 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 51 M 2023-11-02T15:31:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:13.073 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:13.073 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:13.073 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:13.074 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 51 M 2023-11-02T15:31:13.074 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 282 M 2023-11-02T15:31:13.074 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:13.217 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:13.314 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:31:14.994 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:31:15.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_ 1/1 2023-11-02T15:31:15.279 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:15.279 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:15.279 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:15.279 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:15.280 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:15.487 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librbd1-debuginfo 2023-11-02T15:31:16.041 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:08:48 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:16.182 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:16.183 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:16.183 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:16.183 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:16.183 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:16.184 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 51 M 2023-11-02T15:31:16.184 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:16.184 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:16.184 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:16.185 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:16.185 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:16.185 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 51 M 2023-11-02T15:31:16.185 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 292 M 2023-11-02T15:31:16.185 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:33.798 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-debuginfo-18.2.0-1159.gb5a9486b.el9. 2.4 MB/s | 51 MB 00:20 2023-11-02T15:31:33.799 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:33.799 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.4 MB/s | 51 MB 00:20 2023-11-02T15:31:33.800 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:33.803 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:33.804 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:34.137 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:34.137 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:36.608 INFO:teuthology.orchestra.run.smithi174.stdout:librbd1-debuginfo-18.2.0-1159.gb5a9486b.el9.x86 2.5 MB/s | 51 MB 00:20 2023-11-02T15:31:36.608 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:36.609 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.5 MB/s | 51 MB 00:20 2023-11-02T15:31:36.609 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:36.613 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:36.613 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:36.958 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:36.959 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:39.952 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:40.054 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:42.002 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:31:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:42.267 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:42.268 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:42.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd-debuginfo 2023-11-02T15:31:42.814 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:42.921 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:42.978 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:09:13 ago on Thu 02 Nov 2023 03:22:29 PM UTC. 2023-11-02T15:31:43.121 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-11-02T15:31:43.122 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.2 M 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:43.123 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-11-02T15:31:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-11-02T15:31:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-11-02T15:31:43.124 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 1.2 M 2023-11-02T15:31:43.124 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 5.4 M 2023-11-02T15:31:43.125 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-11-02T15:31:43.916 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-debuginfo-18.2.0-1159.gb5a9486b.el9.x86 1.6 MB/s | 1.2 MB 00:00 2023-11-02T15:31:43.917 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:43.917 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.6 MB/s | 1.2 MB 00:00 2023-11-02T15:31:43.917 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-11-02T15:31:43.920 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-11-02T15:31:43.920 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-11-02T15:31:43.961 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-11-02T15:31:43.961 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-11-02T15:31:44.198 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-11-02T15:31:44.297 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:44.723 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:44.956 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:45.003 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:45.004 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:45.004 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-11-02T15:31:45.004 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:45.004 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:31:45.004 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-11-02T15:31:45.199 DEBUG:teuthology.parallel:result is None 2023-11-02T15:31:45.249 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:31:45.249 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:45.249 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:45.250 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:45.250 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:45.250 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:45.445 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-cephfs-debuginfo 2023-11-02T15:31:46.012 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:18 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:46.162 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:46.163 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:46.163 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:46.164 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:46.164 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:46.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 615 k 2023-11-02T15:31:46.164 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:46.164 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 615 k 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.1 M 2023-11-02T15:31:46.165 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:46.460 INFO:teuthology.orchestra.run.smithi174.stdout:python3-cephfs-debuginfo-18.2.0-1159.gb5a9486b. 2.0 MB/s | 615 kB 00:00 2023-11-02T15:31:46.461 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:46.461 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.0 MB/s | 615 kB 00:00 2023-11-02T15:31:46.461 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:46.465 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:46.465 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:46.503 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:46.503 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:46.670 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:46.777 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:47.216 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:47.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9 1/1 2023-11-02T15:31:47.505 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:47.505 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:47.505 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:47.505 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:47.505 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:47.743 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rados-debuginfo 2023-11-02T15:31:48.293 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:20 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:48.434 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:48.435 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:48.435 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:48.435 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:48.435 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:48.435 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 770 k 2023-11-02T15:31:48.436 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:48.436 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:48.436 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:48.436 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:48.436 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:48.437 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 770 k 2023-11-02T15:31:48.437 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.7 M 2023-11-02T15:31:48.437 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:48.913 INFO:teuthology.orchestra.run.smithi174.stdout:python3-rados-debuginfo-18.2.0-1159.gb5a9486b.e 1.6 MB/s | 770 kB 00:00 2023-11-02T15:31:48.914 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:48.914 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.6 MB/s | 770 kB 00:00 2023-11-02T15:31:48.915 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:48.918 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:48.918 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:48.957 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:48.957 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:49.141 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:49.699 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:49.990 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9. 1/1 2023-11-02T15:31:49.991 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:49.991 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:49.991 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:49.991 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:49.992 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:50.197 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rbd-debuginfo 2023-11-02T15:31:50.743 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:22 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:50.884 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:50.885 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:50.885 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:50.885 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:50.885 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:50.886 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.1 M 2023-11-02T15:31:50.886 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:50.886 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:50.886 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:50.886 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:50.887 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:50.887 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 1.1 M 2023-11-02T15:31:50.887 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 4.3 M 2023-11-02T15:31:50.887 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:51.224 INFO:teuthology.orchestra.run.smithi174.stdout:python3-rbd-debuginfo-18.2.0-1159.gb5a9486b.el9 3.3 MB/s | 1.1 MB 00:00 2023-11-02T15:31:51.225 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:51.225 INFO:teuthology.orchestra.run.smithi174.stdout:Total 3.3 MB/s | 1.1 MB 00:00 2023-11-02T15:31:51.226 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:51.229 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:51.229 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:51.268 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:51.268 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:51.589 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:51.985 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:52.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rgw-debuginfo 2023-11-02T15:31:53.008 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:25 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 338 k 2023-11-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 338 k 2023-11-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.0 M 2023-11-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:53.644 INFO:teuthology.orchestra.run.smithi174.stdout:python3-rgw-debuginfo-18.2.0-1159.gb5a9486b.el9 682 kB/s | 338 kB 00:00 2023-11-02T15:31:53.645 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:53.645 INFO:teuthology.orchestra.run.smithi174.stdout:Total 681 kB/s | 338 kB 00:00 2023-11-02T15:31:53.645 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:53.648 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:53.648 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:53.683 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:53.684 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:53.821 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:53.934 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:54.314 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:54.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x8 1/1 2023-11-02T15:31:54.589 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:54.589 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:54.589 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:54.589 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:54.590 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:54.765 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-fuse-debuginfo 2023-11-02T15:31:55.312 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:27 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:55.452 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:55.453 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:55.453 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:55.454 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:55.454 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:55.454 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 507 k 2023-11-02T15:31:55.454 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:55.455 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:55.455 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:55.455 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:55.455 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:55.455 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 507 k 2023-11-02T15:31:55.456 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.5 M 2023-11-02T15:31:55.456 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:31:55.738 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-fuse-debuginfo-18.2.0-1159.gb5a9486b.el9.x8 1.7 MB/s | 507 kB 00:00 2023-11-02T15:31:55.739 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:31:55.739 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.7 MB/s | 507 kB 00:00 2023-11-02T15:31:55.740 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:31:55.743 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:31:55.743 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:31:55.778 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:31:55.779 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:31:55.936 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:31:56.046 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:56.472 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:56.748 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_6 1/1 2023-11-02T15:31:56.748 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:56.749 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:31:56.749 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:31:56.749 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:56.749 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:31:56.943 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-mirror-debuginfo 2023-11-02T15:31:57.486 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:29 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:31:57.628 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:31:57.629 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:31:57.630 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:57.630 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:31:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 51 M 2023-11-02T15:31:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:57.630 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:31:57.631 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:31:57.631 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:31:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:31:57.631 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 51 M 2023-11-02T15:31:57.631 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 282 M 2023-11-02T15:31:57.632 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:32:00.233 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-mirror-debuginfo-18.2.0-1159.gb5a9486b.el9. 19 MB/s | 51 MB 00:02 2023-11-02T15:32:00.234 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:32:00.234 INFO:teuthology.orchestra.run.smithi174.stdout:Total 19 MB/s | 51 MB 00:02 2023-11-02T15:32:00.235 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:32:00.238 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:32:00.239 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:32:00.572 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:32:00.573 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:32:06.270 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:32:06.380 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:32:08.351 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:32:08.644 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86 1/1 2023-11-02T15:32:08.645 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:08.645 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:32:08.645 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:32:08.645 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:08.646 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:32:08.850 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-nbd-debuginfo 2023-11-02T15:32:09.403 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:09:41 ago on Thu 02 Nov 2023 03:22:28 PM UTC. 2023-11-02T15:32:09.544 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-02T15:32:09.545 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:32:09.545 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2023-11-02T15:32:09.545 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:32:09.545 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-02T15:32:09.546 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-debuginfo x86_64 2:18.2.0-1159.gb5a9486b.el9 ceph 1.2 M 2023-11-02T15:32:09.546 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:09.546 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-02T15:32:09.546 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-02T15:32:09.546 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2023-11-02T15:32:09.547 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:09.547 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 1.2 M 2023-11-02T15:32:09.547 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 5.4 M 2023-11-02T15:32:09.547 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-02T15:32:10.110 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd-debuginfo-18.2.0-1159.gb5a9486b.el9.x86 2.2 MB/s | 1.2 MB 00:00 2023-11-02T15:32:10.111 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-02T15:32:10.111 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.2 MB/s | 1.2 MB 00:00 2023-11-02T15:32:10.111 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-02T15:32:10.114 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-02T15:32:10.114 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-02T15:32:10.156 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-02T15:32:10.156 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-02T15:32:10.391 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-02T15:32:10.500 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:32:10.924 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:32:11.198 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 1/1 2023-11-02T15:32:11.198 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:11.198 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-02T15:32:11.198 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-debuginfo-2:18.2.0-1159.gb5a9486b.el9.x86_64 2023-11-02T15:32:11.199 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:11.199 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-02T15:32:11.401 DEBUG:teuthology.parallel:result is None 2023-11-02T15:32:11.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:32:11.544 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-02T15:32:11.565 INFO:teuthology.orchestra.run.smithi174.stdout:18.2.0-1159.gb5a9486b.el9 2023-11-02T15:32:11.566 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1159.gb5a9486b.el9 2023-11-02T15:32:11.567 INFO:teuthology.task.install:The correct ceph version 18.2.0-1159.gb5a9486b is installed. 2023-11-02T15:32:11.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:32:11.922 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-02T15:32:11.948 INFO:teuthology.orchestra.run.smithi175.stdout:18.2.0-1159.gb5a9486b.el9 2023-11-02T15:32:11.949 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1159.gb5a9486b.el9 2023-11-02T15:32:11.949 INFO:teuthology.task.install:The correct ceph version 18.2.0-1159.gb5a9486b is installed. 2023-11-02T15:32:11.949 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-02T15:32:11.950 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:11.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-02T15:32:11.978 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:11.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-02T15:32:12.020 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-02T15:32:12.021 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:12.021 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2023-11-02T15:32:12.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-02T15:32:12.117 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:12.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2023-11-02T15:32:12.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-02T15:32:12.217 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-02T15:32:12.218 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:12.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-02T15:32:12.245 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-02T15:32:12.312 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:12.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-02T15:32:12.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-02T15:32:12.411 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-02T15:32:12.412 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:12.412 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2023-11-02T15:32:12.439 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-02T15:32:12.505 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:12.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2023-11-02T15:32:12.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-02T15:32:12.605 INFO:teuthology.run_tasks:Running task ceph... 2023-11-02T15:32:12.668 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-02T15:32:12.669 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /var/log/ceph 2023-11-02T15:32:12.671 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2023-11-02T15:32:12.700 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-02T15:32:12.700 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-02T15:32:12.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-02T15:32:12.770 INFO:tasks.ceph:Creating extra log directories... 2023-11-02T15:32:12.770 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-02T15:32:12.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-02T15:32:12.847 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-02T15:32:12.848 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-02T15:32:12.848 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599', 'branch': 'wip-yuri6-testing-2023-11-01-0745-reef', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7444599', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2023-11-01-0745-reef'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2023-11-01-0745-reef', 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 49, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 525, 'sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2023-11-01-0745-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081', 'targets': {'smithi174.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBugmfG+rwui8Ts34mPlTCz0WfNTdtwY57m8tJjMFcAuonN4c0UAtSWyRtuAfmNqAPq1XJ7wLMHJ0GnsykGZZe8=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgiUDqKH+Rx2Vp7SONDWsX0Ye5bRtCPgwzqtDCC/9kP7WEpPClEKou7KBljVcDfFamD9gd/+RNBQ/+LSXdMf10='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-02_14:20:05', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-11-02T15:32:12.849 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-02T15:32:12.873 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-02T15:32:12.893 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/run/ceph 2023-11-02T15:32:12.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2023-11-02T15:32:12.960 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:12.961 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2023-11-02T15:32:13.007 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-02T15:32:13.008 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2023-11-02T15:32:13.062 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-02T15:32:13.062 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.063 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-11-02T15:32:13.063 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.063 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.063 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-02 15:24:14.672717452 +0000 2023-11-02T15:32:13.063 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-02 15:21:41.624804031 +0000 2023-11-02T15:32:13.064 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-02 15:21:41.624804031 +0000 2023-11-02T15:32:13.064 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2023-11-02 15:21:41.624804031 +0000 2023-11-02T15:32:13.064 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-02T15:32:13.130 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-02T15:32:13.130 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-02T15:32:13.131 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000287264 s, 1.8 MB/s 2023-11-02T15:32:13.131 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-02T15:32:13.185 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2023-11-02T15:32:13.240 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-02T15:32:13.240 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.240 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-11-02T15:32:13.241 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.241 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.241 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-02 15:24:14.672717452 +0000 2023-11-02T15:32:13.241 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-02 15:21:42.128798098 +0000 2023-11-02T15:32:13.241 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-02 15:21:42.128798098 +0000 2023-11-02T15:32:13.242 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2023-11-02 15:21:42.128798098 +0000 2023-11-02T15:32:13.242 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-02T15:32:13.305 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-02T15:32:13.306 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-02T15:32:13.306 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000225755 s, 2.3 MB/s 2023-11-02T15:32:13.307 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-02T15:32:13.363 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2023-11-02T15:32:13.418 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-02T15:32:13.418 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-11-02T15:32:13.419 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.419 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.419 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-02 15:24:14.672717452 +0000 2023-11-02T15:32:13.419 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-02 15:21:42.621792295 +0000 2023-11-02T15:32:13.419 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-02 15:21:42.621792295 +0000 2023-11-02T15:32:13.420 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2023-11-02 15:21:42.621792295 +0000 2023-11-02T15:32:13.420 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-02T15:32:13.483 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-02T15:32:13.484 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-02T15:32:13.484 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.00020622 s, 2.5 MB/s 2023-11-02T15:32:13.485 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-02T15:32:13.542 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2023-11-02T15:32:13.597 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-02T15:32:13.597 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.597 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-11-02T15:32:13.598 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.598 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.598 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-02 15:24:14.673717440 +0000 2023-11-02T15:32:13.598 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-02 15:21:43.117786456 +0000 2023-11-02T15:32:13.598 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-02 15:21:43.117786456 +0000 2023-11-02T15:32:13.599 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2023-11-02 15:21:43.117786456 +0000 2023-11-02T15:32:13.599 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-02T15:32:13.662 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-02T15:32:13.663 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-02T15:32:13.663 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000164528 s, 3.1 MB/s 2023-11-02T15:32:13.664 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-02T15:32:13.719 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:13.720 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:13.720 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2023-11-02T15:32:13.742 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-02T15:32:13.742 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2023-11-02T15:32:13.800 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-02T15:32:13.800 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.800 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-11-02 15:24:13.338370895 +0000 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-11-02 15:21:43.827116188 +0000 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-11-02 15:21:43.827116188 +0000 2023-11-02T15:32:13.801 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2023-11-02 15:21:43.827116188 +0000 2023-11-02T15:32:13.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-02T15:32:13.867 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-11-02T15:32:13.868 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-11-02T15:32:13.868 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000301029 s, 1.7 MB/s 2023-11-02T15:32:13.869 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-02T15:32:13.927 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2023-11-02T15:32:13.982 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-02T15:32:13.983 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:13.983 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 684 Links: 1 2023-11-02T15:32:13.983 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:13.983 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:13.984 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-11-02 15:24:13.338370895 +0000 2023-11-02T15:32:13.984 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-11-02 15:21:44.310108646 +0000 2023-11-02T15:32:13.984 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-11-02 15:21:44.310108646 +0000 2023-11-02T15:32:13.984 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2023-11-02 15:21:44.310108646 +0000 2023-11-02T15:32:13.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-02T15:32:14.049 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-11-02T15:32:14.050 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-11-02T15:32:14.050 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000189945 s, 2.7 MB/s 2023-11-02T15:32:14.051 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-02T15:32:14.108 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2023-11-02T15:32:14.163 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-02T15:32:14.163 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:14.163 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-11-02T15:32:14.164 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:14.164 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:14.164 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-11-02 15:24:13.339370880 +0000 2023-11-02T15:32:14.164 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-11-02 15:21:44.789101167 +0000 2023-11-02T15:32:14.164 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-11-02 15:21:44.789101167 +0000 2023-11-02T15:32:14.165 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2023-11-02 15:21:44.789101167 +0000 2023-11-02T15:32:14.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-02T15:32:14.229 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-11-02T15:32:14.229 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-11-02T15:32:14.229 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000214553 s, 2.4 MB/s 2023-11-02T15:32:14.230 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-02T15:32:14.287 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2023-11-02T15:32:14.342 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-02T15:32:14.342 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-02T15:32:14.343 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-11-02T15:32:14.343 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-02T15:32:14.343 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-11-02T15:32:14.343 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-11-02 15:24:13.339370880 +0000 2023-11-02T15:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-11-02 15:21:45.264093749 +0000 2023-11-02T15:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-11-02 15:21:45.264093749 +0000 2023-11-02T15:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2023-11-02 15:21:45.264093749 +0000 2023-11-02T15:32:14.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-02T15:32:14.409 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-11-02T15:32:14.409 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-11-02T15:32:14.409 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000188779 s, 2.7 MB/s 2023-11-02T15:32:14.411 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-02T15:32:14.467 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:14.467 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-02T15:32:14.467 INFO:tasks.ceph:Generating config... 2023-11-02T15:32:14.469 INFO:tasks.ceph:[client] debug monc = 20 2023-11-02T15:32:14.469 INFO:tasks.ceph:[client] debug ms = 1 2023-11-02T15:32:14.469 INFO:tasks.ceph:[client] debug objecter = 20 2023-11-02T15:32:14.470 INFO:tasks.ceph:[client] debug rados = 20 2023-11-02T15:32:14.470 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-11-02T15:32:14.470 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-11-02T15:32:14.471 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-11-02T15:32:14.471 INFO:tasks.ceph:[global] ms type = async 2023-11-02T15:32:14.471 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2023-11-02T15:32:14.471 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-02T15:32:14.472 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-02T15:32:14.472 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-02T15:32:14.472 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-02T15:32:14.472 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-02T15:32:14.473 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-11-02T15:32:14.473 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2023-11-02T15:32:14.473 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-11-02T15:32:14.474 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-11-02T15:32:14.474 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-11-02T15:32:14.474 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-11-02T15:32:14.475 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2023-11-02T15:32:14.475 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-11-02T15:32:14.475 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-02T15:32:14.475 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-02T15:32:14.476 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-02T15:32:14.476 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-02T15:32:14.476 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-11-02T15:32:14.476 INFO:tasks.ceph:[osd] debug bluefs = 1 2023-11-02T15:32:14.477 INFO:tasks.ceph:[osd] debug bluestore = 1 2023-11-02T15:32:14.477 INFO:tasks.ceph:[osd] debug monc = 20 2023-11-02T15:32:14.477 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-02T15:32:14.478 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-02T15:32:14.478 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-02T15:32:14.478 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-02T15:32:14.478 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-02T15:32:14.479 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-02T15:32:14.479 INFO:tasks.ceph:[osd] osd class default list = * 2023-11-02T15:32:14.479 INFO:tasks.ceph:[osd] osd class load list = * 2023-11-02T15:32:14.480 INFO:tasks.ceph:[osd] osd client watch timeout = 120 2023-11-02T15:32:14.480 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-11-02T15:32:14.480 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-11-02T15:32:14.481 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-02T15:32:14.481 INFO:tasks.ceph:[osd] osd fast shutdown = False 2023-11-02T15:32:14.481 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-11-02T15:32:14.482 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-11-02T15:32:14.482 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2023-11-02T15:32:14.482 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-02T15:32:14.483 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-11-02T15:32:14.483 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-11-02T15:32:14.483 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-11-02T15:32:14.484 INFO:tasks.ceph:Setting up mon.a... 2023-11-02T15:32:14.484 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-02T15:32:14.536 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.keyring 2023-11-02T15:32:14.539 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-02T15:32:14.585 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-02T15:32:14.612 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.174'), ('mon.c', '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]'), ('mon.b', '172.21.15.175')] 2023-11-02T15:32:14.612 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790],172.21.15.175', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-11-02T15:32:14.614 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:14.614 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-02T15:32:14.667 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.174 --addv c '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]' --add b 172.21.15.175 --print /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:14.752 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:14.752 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: generated fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:32:14.752 INFO:teuthology.orchestra.run.smithi174.stdout:setting min_mon_release = pacific 2023-11-02T15:32:14.752 INFO:teuthology.orchestra.run.smithi174.stdout:epoch 0 2023-11-02T15:32:14.753 INFO:teuthology.orchestra.run.smithi174.stdout:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:32:14.753 INFO:teuthology.orchestra.run.smithi174.stdout:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:32:14.753 INFO:teuthology.orchestra.run.smithi174.stdout:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:32:14.753 INFO:teuthology.orchestra.run.smithi174.stdout:min_mon_release 16 (pacific) 2023-11-02T15:32:14.754 INFO:teuthology.orchestra.run.smithi174.stdout:election_strategy: 1 2023-11-02T15:32:14.754 INFO:teuthology.orchestra.run.smithi174.stdout:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:32:14.754 INFO:teuthology.orchestra.run.smithi174.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:32:14.754 INFO:teuthology.orchestra.run.smithi174.stdout:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:32:14.754 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-02T15:32:14.756 DEBUG:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-02T15:32:14.770 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 71bb545f-71ad-46c8-9016-794bb3baf3fa... 2023-11-02T15:32:14.771 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-02T15:32:14.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-02T15:32:14.860 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2023-11-02T15:32:14.860 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2023-11-02T15:32:14.860 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-02T15:32:14.861 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2023-11-02T15:32:14.861 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.861 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2023-11-02T15:32:14.861 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.861 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2023-11-02T15:32:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2023-11-02T15:32:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2023-11-02T15:32:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2023-11-02T15:32:14.863 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2023-11-02T15:32:14.863 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.863 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-02T15:32:14.863 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2023-11-02T15:32:14.863 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.864 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2023-11-02T15:32:14.864 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-02T15:32:14.864 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.864 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2023-11-02T15:32:14.864 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2023-11-02T15:32:14.865 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2023-11-02T15:32:14.866 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2023-11-02T15:32:14.866 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-02T15:32:14.866 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-02T15:32:14.866 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2023-11-02T15:32:14.866 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.867 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-02T15:32:14.867 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.867 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2023-11-02T15:32:14.867 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.867 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790],172.21.15.175" 2023-11-02T15:32:14.868 INFO:teuthology.orchestra.run.smithi174.stdout: mon client directed command retry = 5 2023-11-02T15:32:14.869 INFO:teuthology.orchestra.run.smithi174.stdout: mon election default strategy = 1 2023-11-02T15:32:14.869 INFO:teuthology.orchestra.run.smithi174.stdout: ms inject socket failures = 5000 2023-11-02T15:32:14.869 INFO:teuthology.orchestra.run.smithi174.stdout: ms type = async 2023-11-02T15:32:14.869 INFO:teuthology.orchestra.run.smithi174.stdout: osd heartbeat grace = 80 2023-11-02T15:32:14.869 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:32:14.870 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.870 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2023-11-02T15:32:14.870 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2023-11-02T15:32:14.870 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.870 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2023-11-02T15:32:14.871 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.872 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2023-11-02T15:32:14.872 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2023-11-02T15:32:14.872 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2023-11-02T15:32:14.872 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.872 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2023-11-02T15:32:14.873 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2023-11-02T15:32:14.873 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.873 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2023-11-02T15:32:14.873 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.873 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2023-11-02T15:32:14.874 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.874 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2023-11-02T15:32:14.874 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.874 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2023-11-02T15:32:14.875 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2023-11-02T15:32:14.875 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.875 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2023-11-02T15:32:14.876 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2023-11-02T15:32:14.876 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2023-11-02T15:32:14.876 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2023-11-02T15:32:14.876 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2023-11-02T15:32:14.876 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore zero block detection = True 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: debug monc = 20 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-02T15:32:14.877 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2023-11-02T15:32:14.878 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 10 2023-11-02T15:32:14.878 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2023-11-02T15:32:14.878 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: osd class default list = * 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: osd class load list = * 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: osd client watch timeout = 120 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify cached snaps = True 2023-11-02T15:32:14.879 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify missing on start = True 2023-11-02T15:32:14.880 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2023-11-02T15:32:14.880 INFO:teuthology.orchestra.run.smithi174.stdout: osd fast shutdown = False 2023-11-02T15:32:14.880 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock override recovery settings = True 2023-11-02T15:32:14.880 INFO:teuthology.orchestra.run.smithi174.stdout: osd memory target = 1610612736 2023-11-02T15:32:14.881 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2023-11-02T15:32:14.881 INFO:teuthology.orchestra.run.smithi174.stdout: osd op queue = debug_random 2023-11-02T15:32:14.882 INFO:teuthology.orchestra.run.smithi174.stdout: osd op queue cut off = debug_random 2023-11-02T15:32:14.882 INFO:teuthology.orchestra.run.smithi174.stdout: osd shutdown pgref assert = True 2023-11-02T15:32:14.882 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.882 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2023-11-02T15:32:14.883 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-02T15:32:14.883 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2023-11-02T15:32:14.883 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-02T15:32:14.883 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-02T15:32:14.883 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-02T15:32:14.884 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-02T15:32:14.884 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2023-11-02T15:32:14.884 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.884 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2023-11-02T15:32:14.884 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2023-11-02T15:32:14.885 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-02T15:32:14.886 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2023-11-02T15:32:14.886 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2023-11-02T15:32:14.886 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-02T15:32:14.886 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.886 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-02T15:32:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2023-11-02T15:32:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2023-11-02T15:32:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2023-11-02T15:32:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.888 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2023-11-02T15:32:14.889 INFO:teuthology.orchestra.run.smithi174.stdout: mon min osdmap epochs = 50 2023-11-02T15:32:14.889 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd crush smoke test = False 2023-11-02T15:32:14.889 INFO:teuthology.orchestra.run.smithi174.stdout: mon osdmap full prune interval = 2 2023-11-02T15:32:14.889 INFO:teuthology.orchestra.run.smithi174.stdout: mon osdmap full prune min = 15 2023-11-02T15:32:14.889 INFO:teuthology.orchestra.run.smithi174.stdout: mon osdmap full prune txsize = 2 2023-11-02T15:32:14.890 INFO:teuthology.orchestra.run.smithi174.stdout: mon scrub interval = 300 2023-11-02T15:32:14.890 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on pool no app = False 2023-11-02T15:32:14.890 INFO:teuthology.orchestra.run.smithi174.stdout: paxos service trim min = 10 2023-11-02T15:32:14.890 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:32:14.890 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-02T15:32:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: debug monc = 20 2023-11-02T15:32:14.892 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-02T15:32:14.892 INFO:teuthology.orchestra.run.smithi174.stdout: debug objecter = 20 2023-11-02T15:32:14.892 INFO:teuthology.orchestra.run.smithi174.stdout: debug rados = 20 2023-11-02T15:32:14.892 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2023-11-02T15:32:14.892 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2023-11-02T15:32:14.893 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2023-11-02T15:32:14.894 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2023-11-02T15:32:14.895 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2023-11-02T15:32:14.895 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-02T15:32:14.895 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2023-11-02T15:32:14.895 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.895 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2023-11-02T15:32:14.896 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.897 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2023-11-02T15:32:14.897 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2023-11-02T15:32:14.897 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.897 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-02T15:32:14.897 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2023-11-02T15:32:14.898 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.898 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2023-11-02T15:32:14.898 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-02T15:32:14.898 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.898 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2023-11-02T15:32:14.899 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2023-11-02T15:32:14.900 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2023-11-02T15:32:14.900 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2023-11-02T15:32:14.900 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-02T15:32:14.900 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-02T15:32:14.900 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2023-11-02T15:32:14.901 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.901 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-02T15:32:14.901 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.901 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2023-11-02T15:32:14.901 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2023-11-02T15:32:14.902 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790],172.21.15.175" 2023-11-02T15:32:14.903 INFO:teuthology.orchestra.run.smithi175.stdout: mon client directed command retry = 5 2023-11-02T15:32:14.903 INFO:teuthology.orchestra.run.smithi175.stdout: mon election default strategy = 1 2023-11-02T15:32:14.903 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject socket failures = 5000 2023-11-02T15:32:14.903 INFO:teuthology.orchestra.run.smithi175.stdout: ms type = async 2023-11-02T15:32:14.903 INFO:teuthology.orchestra.run.smithi175.stdout: osd heartbeat grace = 80 2023-11-02T15:32:14.904 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:32:14.904 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.904 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2023-11-02T15:32:14.904 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2023-11-02T15:32:14.904 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.905 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2023-11-02T15:32:14.905 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2023-11-02T15:32:14.905 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2023-11-02T15:32:14.905 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.905 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2023-11-02T15:32:14.906 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.907 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2023-11-02T15:32:14.907 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2023-11-02T15:32:14.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.907 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2023-11-02T15:32:14.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.908 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2023-11-02T15:32:14.908 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.908 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2023-11-02T15:32:14.908 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.908 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2023-11-02T15:32:14.909 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2023-11-02T15:32:14.910 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2023-11-02T15:32:14.910 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2023-11-02T15:32:14.910 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore zero block detection = True 2023-11-02T15:32:14.910 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1 2023-11-02T15:32:14.910 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1 2023-11-02T15:32:14.911 INFO:teuthology.orchestra.run.smithi175.stdout: debug monc = 20 2023-11-02T15:32:14.911 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-11-02T15:32:14.911 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2023-11-02T15:32:14.911 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 10 2023-11-02T15:32:14.911 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: osd class default list = * 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: osd class load list = * 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: osd client watch timeout = 120 2023-11-02T15:32:14.912 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify cached snaps = True 2023-11-02T15:32:14.913 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify missing on start = True 2023-11-02T15:32:14.913 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2023-11-02T15:32:14.913 INFO:teuthology.orchestra.run.smithi175.stdout: osd fast shutdown = False 2023-11-02T15:32:14.913 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock override recovery settings = True 2023-11-02T15:32:14.913 INFO:teuthology.orchestra.run.smithi175.stdout: osd memory target = 1610612736 2023-11-02T15:32:14.914 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2023-11-02T15:32:14.914 INFO:teuthology.orchestra.run.smithi175.stdout: osd op queue = debug_random 2023-11-02T15:32:14.914 INFO:teuthology.orchestra.run.smithi175.stdout: osd op queue cut off = debug_random 2023-11-02T15:32:14.914 INFO:teuthology.orchestra.run.smithi175.stdout: osd shutdown pgref assert = True 2023-11-02T15:32:14.914 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.915 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2023-11-02T15:32:14.915 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-11-02T15:32:14.915 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2023-11-02T15:32:14.916 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2023-11-02T15:32:14.916 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2023-11-02T15:32:14.916 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2023-11-02T15:32:14.916 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2023-11-02T15:32:14.917 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2023-11-02T15:32:14.917 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.917 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2023-11-02T15:32:14.917 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-11-02T15:32:14.917 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2023-11-02T15:32:14.918 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2023-11-02T15:32:14.918 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2023-11-02T15:32:14.918 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2023-11-02T15:32:14.918 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2023-11-02T15:32:14.918 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2023-11-02T15:32:14.919 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2023-11-02T15:32:14.919 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2023-11-02T15:32:14.919 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2023-11-02T15:32:14.919 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.919 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-02T15:32:14.920 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2023-11-02T15:32:14.920 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2023-11-02T15:32:14.920 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.921 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2023-11-02T15:32:14.921 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-02T15:32:14.921 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-02T15:32:14.921 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.921 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2023-11-02T15:32:14.922 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2023-11-02T15:32:14.922 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.922 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2023-11-02T15:32:14.922 INFO:teuthology.orchestra.run.smithi175.stdout: mon min osdmap epochs = 50 2023-11-02T15:32:14.922 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd crush smoke test = False 2023-11-02T15:32:14.923 INFO:teuthology.orchestra.run.smithi175.stdout: mon osdmap full prune interval = 2 2023-11-02T15:32:14.923 INFO:teuthology.orchestra.run.smithi175.stdout: mon osdmap full prune min = 15 2023-11-02T15:32:14.923 INFO:teuthology.orchestra.run.smithi175.stdout: mon osdmap full prune txsize = 2 2023-11-02T15:32:14.923 INFO:teuthology.orchestra.run.smithi175.stdout: mon scrub interval = 300 2023-11-02T15:32:14.923 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on pool no app = False 2023-11-02T15:32:14.924 INFO:teuthology.orchestra.run.smithi175.stdout: paxos service trim min = 10 2023-11-02T15:32:14.924 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-11-02T15:32:14.924 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2023-11-02T15:32:14.924 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2023-11-02T15:32:14.924 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2023-11-02T15:32:14.925 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2023-11-02T15:32:14.925 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-02T15:32:14.925 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-02T15:32:14.925 INFO:teuthology.orchestra.run.smithi175.stdout: debug monc = 20 2023-11-02T15:32:14.926 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-11-02T15:32:14.926 INFO:teuthology.orchestra.run.smithi175.stdout: debug objecter = 20 2023-11-02T15:32:14.926 INFO:teuthology.orchestra.run.smithi175.stdout: debug rados = 20 2023-11-02T15:32:14.926 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2023-11-02T15:32:14.927 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2023-11-02T15:32:14.927 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2023-11-02T15:32:14.927 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-02T15:32:14.928 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-02T15:32:14.989 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-02T15:32:14.989 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:14.990 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-02T15:32:15.005 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:15.005 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-02T15:32:15.060 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:32:15.060 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:15.061 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-02T15:32:15.061 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-02T15:32:15.139 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:15.139 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:15.196 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:32:15.196 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:15.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-02T15:32:15.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-02T15:32:15.239 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:15.240 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:15.295 INFO:tasks.ceph:Setting up mon nodes... 2023-11-02T15:32:15.296 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-02T15:32:15.296 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-02T15:32:15.351 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-02T15:32:15.354 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-02T15:32:15.419 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-02T15:32:15.422 INFO:tasks.ceph:Setting up mds nodes... 2023-11-02T15:32:15.422 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-02T15:32:15.423 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-02T15:32:15.465 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-02T15:32:15.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-02T15:32:15.523 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-02T15:32:15.538 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-02T15:32:15.539 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-02T15:32:15.539 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-02T15:32:15.566 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:15.566 INFO:tasks.ceph:role: osd.0 2023-11-02T15:32:15.567 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:32:15.567 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-02T15:32:15.640 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:15.640 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:15.640 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:15.641 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:15.641 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:15.641 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:15.641 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:15.641 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:15.642 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:15.642 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:16.144 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-02T15:32:16.146 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-02T15:32:16.146 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-02T15:32:16.271 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-02T15:32:16.307 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-02T15:32:16.376 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:16.376 INFO:tasks.ceph:role: osd.1 2023-11-02T15:32:16.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:32:16.377 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-02T15:32:16.452 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:16.453 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:16.454 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:16.454 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:16.454 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:16.454 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:16.906 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-02T15:32:16.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-02T15:32:16.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-02T15:32:17.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-02T15:32:17.035 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-02T15:32:17.103 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:17.103 INFO:tasks.ceph:role: osd.2 2023-11-02T15:32:17.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:32:17.104 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-02T15:32:17.178 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:17.178 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:17.178 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:17.179 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:17.179 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:17.179 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:17.179 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:17.179 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:17.180 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:17.180 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-02T15:32:17.672 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-02T15:32:17.673 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-02T15:32:17.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-02T15:32:17.846 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-02T15:32:17.915 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:17.916 INFO:tasks.ceph:role: osd.3 2023-11-02T15:32:17.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T15:32:17.916 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-02T15:32:17.991 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:17.991 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:17.991 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:17.991 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:17.992 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:17.992 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:17.992 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:17.992 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:17.992 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:17.993 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:18.516 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-02T15:32:18.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-02T15:32:18.518 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-02T15:32:18.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-02T15:32:18.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:18.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.798+0000 7f0f27c8c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-02T15:32:18.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.799+0000 7f0f27c8c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-02T15:32:18.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.799+0000 7f0f27c8c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-02T15:32:18.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.799+0000 7f0f27c8c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-02T15:32:18.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.799+0000 7f0f27c8c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-02T15:32:18.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:18.799+0000 7f0f27c8c500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-02T15:32:21.643 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-02T15:32:21.674 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:21.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.764+0000 7fb0bd216500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-02T15:32:21.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.764+0000 7fb0bd216500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-02T15:32:21.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.764+0000 7fb0bd216500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-02T15:32:21.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.764+0000 7fb0bd216500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-02T15:32:21.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.764+0000 7fb0bd216500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-02T15:32:21.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:21.765+0000 7fb0bd216500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-02T15:32:24.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-02T15:32:24.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:24.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.732+0000 7f49e0818500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-02T15:32:24.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.732+0000 7f49e0818500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-02T15:32:24.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.732+0000 7f49e0818500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-02T15:32:24.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.732+0000 7f49e0818500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-02T15:32:24.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.732+0000 7f49e0818500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-02T15:32:24.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:24.733+0000 7f49e0818500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-02T15:32:27.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-02T15:32:27.623 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:27.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.711+0000 7f62335bc500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-02T15:32:27.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.711+0000 7f62335bc500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-02T15:32:27.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.711+0000 7f62335bc500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-02T15:32:27.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.711+0000 7f62335bc500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-02T15:32:27.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.711+0000 7f62335bc500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-02T15:32:27.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:27.712+0000 7f62335bc500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-02T15:32:30.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-02T15:32:30.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-02T15:32:30.628 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:30.629 INFO:tasks.ceph:role: osd.4 2023-11-02T15:32:30.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:32:30.630 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-02T15:32:30.706 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:30.707 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:30.708 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:30.708 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:30.708 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:31.219 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-11-02T15:32:31.221 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-11-02T15:32:31.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-02T15:32:31.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-02T15:32:31.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-02T15:32:31.454 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:31.455 INFO:tasks.ceph:role: osd.5 2023-11-02T15:32:31.455 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:32:31.455 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-02T15:32:31.530 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:31.531 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:31.531 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:31.531 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:31.532 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:31.986 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-11-02T15:32:31.988 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-11-02T15:32:31.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-02T15:32:32.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-02T15:32:32.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-02T15:32:32.192 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:32.192 INFO:tasks.ceph:role: osd.6 2023-11-02T15:32:32.192 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:32:32.193 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-02T15:32:32.266 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:32.267 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:32.267 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:32.267 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:32.267 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:32.268 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:32.268 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:32.268 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:32.268 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:32.269 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:32.754 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-11-02T15:32:32.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-11-02T15:32:32.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-02T15:32:32.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-02T15:32:32.945 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-02T15:32:33.014 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-02T15:32:33.014 INFO:tasks.ceph:role: osd.7 2023-11-02T15:32:33.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T15:32:33.015 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-02T15:32:33.090 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-02T15:32:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-02T15:32:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-02T15:32:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-02T15:32:33.091 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-02T15:32:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-11-02T15:32:33.091 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-02T15:32:33.092 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-02T15:32:33.092 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-02T15:32:33.092 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-02T15:32:33.605 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-11-02T15:32:33.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-11-02T15:32:33.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-02T15:32:33.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-02T15:32:33.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:33.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.860+0000 7fccd53fb500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-02T15:32:33.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.860+0000 7fccd53fb500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-02T15:32:33.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.860+0000 7fccd53fb500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-02T15:32:33.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.860+0000 7fccd53fb500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-02T15:32:33.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.860+0000 7fccd53fb500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-02T15:32:33.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:33.861+0000 7fccd53fb500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-02T15:32:36.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-02T15:32:36.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:36.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.834+0000 7f242b8a3500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-02T15:32:36.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.834+0000 7f242b8a3500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-02T15:32:36.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.834+0000 7f242b8a3500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-02T15:32:36.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.835+0000 7f242b8a3500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-02T15:32:36.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.835+0000 7f242b8a3500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-02T15:32:36.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:36.835+0000 7f242b8a3500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-02T15:32:39.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-02T15:32:39.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:39.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.803+0000 7f974f21a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-02T15:32:39.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.804+0000 7f974f21a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-02T15:32:39.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.804+0000 7f974f21a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-02T15:32:39.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.804+0000 7f974f21a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-02T15:32:39.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.804+0000 7f974f21a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-02T15:32:39.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:39.804+0000 7f974f21a500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-02T15:32:42.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-02T15:32:42.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:42.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.781+0000 7f4ed18fa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-02T15:32:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.781+0000 7f4ed18fa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-02T15:32:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.781+0000 7f4ed18fa500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-02T15:32:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.781+0000 7f4ed18fa500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-02T15:32:42.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.781+0000 7f4ed18fa500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-02T15:32:42.785 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:32:42.782+0000 7f4ed18fa500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-02T15:32:45.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-02T15:32:45.662 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-02T15:32:45.663 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:45.663 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-02T15:32:45.696 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:45.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-02T15:32:45.765 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:45.766 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-02T15:32:45.834 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:45.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-02T15:32:45.902 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:45.903 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-02T15:32:45.970 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:45.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-02T15:32:46.002 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:46.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-02T15:32:46.074 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:46.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-02T15:32:46.149 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:46.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-02T15:32:46.290 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:46.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-02T15:32:46.361 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:32:46.361 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-02T15:32:46.378 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:32:46.379 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-02T15:32:46.418 INFO:tasks.ceph:Adding keys to all mons... 2023-11-02T15:32:46.418 DEBUG:teuthology.orchestra.run.smithi174:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-02T15:32:46.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-02T15:32:46.462 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.y] 2023-11-02T15:32:46.462 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/wENlOFrrFBAAl+BR4gIkEQ6V9APql+9sGQ== 2023-11-02T15:32:46.463 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.0] 2023-11-02T15:32:46.463 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQACwUNl2QylLxAA4GSrTbrNgPLqviB9stcjNA== 2023-11-02T15:32:46.463 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.1] 2023-11-02T15:32:46.463 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAFwUNld2WbLRAA8ZzN+wf7oeCkOZqiJIypMA== 2023-11-02T15:32:46.463 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.2] 2023-11-02T15:32:46.464 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIwUNlquq1KxAAfPEbep3+qBpP/tg589Vwww== 2023-11-02T15:32:46.464 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.3] 2023-11-02T15:32:46.464 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQALwUNlhOh2KhAAyQKCw3c1xRvoExQqV9LV1A== 2023-11-02T15:32:46.464 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.x] 2023-11-02T15:32:46.464 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/wENl1ev3GBAAYHdsoHkyaaOeuhEpp2hkgw== 2023-11-02T15:32:46.465 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.4] 2023-11-02T15:32:46.465 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQARwUNleLVcMxAAqpr3e1ovgUnTGlhLrnLOnA== 2023-11-02T15:32:46.465 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.5] 2023-11-02T15:32:46.465 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAUwUNle+XSMRAAMaXsTsBqXTXbnfe+bGXheg== 2023-11-02T15:32:46.465 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.6] 2023-11-02T15:32:46.466 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAXwUNlMxH7LxAAv4WBWDWncSZ3sobR0jL2lA== 2023-11-02T15:32:46.466 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.7] 2023-11-02T15:32:46.466 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAawUNltfKnLhAAyoepFycVi/J6zyFX7moJvg== 2023-11-02T15:32:46.466 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2023-11-02T15:32:46.466 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/wENlEumzGxAAyQCBn5Q1cKRuK/viy8FsVQ== 2023-11-02T15:32:46.467 INFO:teuthology.orchestra.run.smithi174.stdout:[client.1] 2023-11-02T15:32:46.467 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/wENlSustHxAA5sgCD4Y48aPrlkKpUKuZyw== 2023-11-02T15:32:46.486 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2023-11-02T15:32:46.486 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD/wENlOFrrFBAAl+BR4gIkEQ6V9APql+9sGQ== 2023-11-02T15:32:46.489 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2023-11-02T15:32:46.489 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQACwUNl2QylLxAA4GSrTbrNgPLqviB9stcjNA== 2023-11-02T15:32:46.490 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2023-11-02T15:32:46.490 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAFwUNld2WbLRAA8ZzN+wf7oeCkOZqiJIypMA== 2023-11-02T15:32:46.490 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2023-11-02T15:32:46.490 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAIwUNlquq1KxAAfPEbep3+qBpP/tg589Vwww== 2023-11-02T15:32:46.490 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2023-11-02T15:32:46.491 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQALwUNlhOh2KhAAyQKCw3c1xRvoExQqV9LV1A== 2023-11-02T15:32:46.491 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2023-11-02T15:32:46.491 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD/wENl1ev3GBAAYHdsoHkyaaOeuhEpp2hkgw== 2023-11-02T15:32:46.491 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2023-11-02T15:32:46.491 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQARwUNleLVcMxAAqpr3e1ovgUnTGlhLrnLOnA== 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAUwUNle+XSMRAAMaXsTsBqXTXbnfe+bGXheg== 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAXwUNlMxH7LxAAv4WBWDWncSZ3sobR0jL2lA== 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2023-11-02T15:32:46.492 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAawUNltfKnLhAAyoepFycVi/J6zyFX7moJvg== 2023-11-02T15:32:46.493 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2023-11-02T15:32:46.493 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD/wENlEumzGxAAyQCBn5Q1cKRuK/viy8FsVQ== 2023-11-02T15:32:46.493 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2023-11-02T15:32:46.493 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD/wENlSustHxAA5sgCD4Y48aPrlkKpUKuZyw== 2023-11-02T15:32:46.494 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-02T15:32:46.511 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-02T15:32:46.589 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.591 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.643 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.698 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.752 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.806 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-02T15:32:46.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-02T15:32:46.859 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.968 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:46.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:47.022 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:47.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-02T15:32:47.076 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-02T15:32:47.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-02T15:32:47.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-02T15:32:47.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-02T15:32:47.189 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-02T15:32:47.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-02T15:32:47.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-02T15:32:47.314 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-02T15:32:47.341 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-02T15:32:47.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-02T15:32:47.505 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-02T15:32:47.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-02T15:32:47.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-02T15:32:47.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-02T15:32:47.687 DEBUG:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:47.690 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-02T15:32:47.742 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-02T15:32:47.743 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-11-02T15:32:47.743 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-02T15:32:47.744 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-11-02T15:32:47.746 INFO:tasks.ceph.mon.a:Started 2023-11-02T15:32:47.747 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-11-02T15:32:47.747 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-02T15:32:47.747 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-11-02T15:32:47.749 INFO:tasks.ceph.mon.c:Started 2023-11-02T15:32:47.750 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-11-02T15:32:47.750 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-02T15:32:47.751 DEBUG:teuthology.orchestra.run.smithi175:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-11-02T15:32:47.785 INFO:tasks.ceph.mon.b:Started 2023-11-02T15:32:47.785 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-02T15:32:47.786 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-02T15:32:47.787 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2023-11-02T15:32:47.789 INFO:tasks.ceph.mgr.y:Started 2023-11-02T15:32:47.790 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-02T15:32:47.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2023-11-02T15:32:47.794 INFO:tasks.ceph.mgr.x:Started 2023-11-02T15:32:47.794 INFO:tasks.ceph:Setting crush tunables to default 2023-11-02T15:32:47.794 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd crush tunables default 2023-11-02T15:32:47.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.895+0000 7fecff100640 10 monclient: get_monmap_and_config 2023-11-02T15:32:47.897 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.895+0000 7fecff100640 10 monclient: build_initial_monmap 2023-11-02T15:32:47.897 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.895+0000 7fecff100640 10 monclient: monmap: 2023-11-02T15:32:47.897 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:32:47.897 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:32:47.897 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:47.896135+0000 2023-11-02T15:32:47.898 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:47.896135+0000 2023-11-02T15:32:47.898 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:32:47.898 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:32:47.898 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:32:47.898 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:32:47.901 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:32:47.901 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:32:47.901 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:32:47.902 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:32:47.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.895+0000 7fecff100640 1 Processor -- start 2023-11-02T15:32:47.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 1 -- start start 2023-11-02T15:32:47.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient: init 2023-11-02T15:32:47.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient: _reopen_session rank -1 2023-11-02T15:32:47.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2023-11-02T15:32:47.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecf80b4570 addr v1:172.21.15.174:6789/0 2023-11-02T15:32:47.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fecf80b0ed0 addr v1:172.21.15.175:6789/0 2023-11-02T15:32:47.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:47.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): picked mon.noname-b con 0x7fecf8144d40 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:32:47.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80a9780 con 0x7fecf80b4570 2023-11-02T15:32:47.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf81451d0 con 0x7fecf80b0ed0 2023-11-02T15:32:47.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:47.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecf8144d40 2023-11-02T15:32:47.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): _renew_subs 2023-11-02T15:32:47.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecff100640 10 monclient(hunting): authenticate will time out at 1295.238291s 2023-11-02T15:32:47.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecfce9c640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecf80b4570 legacy=0x7fecf80a9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:47.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecfd69d640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecf813ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:47.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:47.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:47.896+0000 7fecf7fff640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecf80b0ed0 legacy=0x7fecf813e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.175:6789/0 2023-11-02T15:32:48.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.097+0000 7fecfce9c640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecf80b4570 legacy=0x7fecf80a9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:48.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.097+0000 7fecfd69d640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecf813ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:48.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.097+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:48.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.097+0000 7fecf7fff640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecf80b0ed0 legacy=0x7fecf813e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.175:6789/0 2023-11-02T15:32:48.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.498+0000 7fecfce9c640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecf80b4570 legacy=0x7fecf80a9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:48.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.498+0000 7fecfd69d640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecf813ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:48.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.498+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:48.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:48.498+0000 7fecf7fff640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecf80b0ed0 legacy=0x7fecf813e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.175:6789/0 2023-11-02T15:32:49.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:49.299+0000 7fecfce9c640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecf80b4570 legacy=0x7fecf80a9070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:49.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:49.299+0000 7fecfd69d640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecf813ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:49.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:49.299+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:49.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:49.299+0000 7fecf7fff640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecf80b0ed0 legacy=0x7fecf813e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.175:6789/0 2023-11-02T15:32:50.898 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:32:50.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:32:50.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:32:50.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:32:50.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecf813ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecf813ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecf80b0ed0 legacy=0x7fecf813e520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecf80b4570 legacy=0x7fecf80a9070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient: _add_conns ranks=[3,4] 2023-11-02T15:32:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:50.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc003700 addr v2:172.21.15.174:3300/0 2023-11-02T15:32:50.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:50.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c con 0x7fecdc006270 addr v2:172.21.15.175:3300/0 2023-11-02T15:32:50.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:50.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecdc003700 2023-11-02T15:32:50.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:50.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc008df0 con 0x7fecdc006270 2023-11-02T15:32:50.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:32:50.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecfce9c640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 msgr2=0x7fecdc005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:50.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecfce9c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:50.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf7fff640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 msgr2=0x7fecdc008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.175:3300/0 2023-11-02T15:32:50.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:50.896+0000 7fecf7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:51.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.097+0000 7fecfce9c640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 msgr2=0x7fecdc005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:51.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.097+0000 7fecfce9c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:51.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.097+0000 7fecf7fff640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 msgr2=0x7fecdc008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.175:3300/0 2023-11-02T15:32:51.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.097+0000 7fecf7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:51.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.498+0000 7fecfce9c640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 msgr2=0x7fecdc005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:51.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.498+0000 7fecfce9c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:51.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.498+0000 7fecf7fff640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 msgr2=0x7fecdc008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.175:3300/0 2023-11-02T15:32:51.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:51.498+0000 7fecf7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:52.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:52.299+0000 7fecfce9c640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 msgr2=0x7fecdc005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:52.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:52.299+0000 7fecfce9c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:52.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:52.299+0000 7fecf7fff640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 msgr2=0x7fecdc008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.175:3300/0 2023-11-02T15:32:52.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:52.299+0000 7fecf7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:32:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:32:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:32:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:32:53.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 msgr2=0x7fecdc008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:53.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc006270 0x7fecdc008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:53.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 msgr2=0x7fecdc005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:53.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003700 0x7fecdc005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:53.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient: _add_conns ranks=[2,3,0,4,1] 2023-11-02T15:32:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecfd69d640 1 -- reap_dead start 2023-11-02T15:32:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-b con 0x7fecf8144d40 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:32:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc003b70 addr v2:172.21.15.174:3300/0 2023-11-02T15:32:53.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecdc006c60 addr v1:172.21.15.174:6789/0 2023-11-02T15:32:53.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80aa4b0 con 0x7fecdc006c60 2023-11-02T15:32:53.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:53.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecdc003b70 2023-11-02T15:32:53.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:53.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecfce9c640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:53.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc008df0 con 0x7fecf8144d40 2023-11-02T15:32:53.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.896+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:32:53.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.897+0000 7fecfce9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:53.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.897+0000 7fecf7fff640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:53.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.897+0000 7fecfd69d640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecdc006c60 legacy=0x7fecdc0070c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:53.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:53.897+0000 7fecf7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:54.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.098+0000 7fecfce9c640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:54.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.098+0000 7fecfd69d640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecdc006c60 legacy=0x7fecdc0070c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:54.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.098+0000 7fecf7fff640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:54.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.098+0000 7fecfce9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:54.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.098+0000 7fecf7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:54.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.499+0000 7fecfce9c640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:54.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.499+0000 7fecf7fff640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:54.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.499+0000 7fecfd69d640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecdc006c60 legacy=0x7fecdc0070c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:54.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.499+0000 7fecfce9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:54.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:54.499+0000 7fecf7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:55.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:55.300+0000 7fecf7fff640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3300/0 2023-11-02T15:32:55.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:55.300+0000 7fecfd69d640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecdc006c60 legacy=0x7fecdc0070c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.174:6789/0 2023-11-02T15:32:55.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:55.300+0000 7fecfce9c640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.174:3301/0 2023-11-02T15:32:55.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:55.300+0000 7fecf7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:55.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:55.300+0000 7fecfce9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:55.582 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:00:07.678 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:55.699 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:00:07.852 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:55.707 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:00:07.860 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:55.902 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:00:07.996 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:56.014 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:00:08.168 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:56.024 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:00:08.177 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:32:56.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:32:56.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:32:56.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 -- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 -- >> v1:172.21.15.174:6789/0 conn(0x7fecdc006c60 legacy=0x7fecdc0070c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient: _add_conns ranks=[1,4] 2023-11-02T15:32:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fecdc008880 addr v1:172.21.15.175:6789/0 2023-11-02T15:32:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c con 0x7fecdc010ea0 addr v2:172.21.15.175:3300/0 2023-11-02T15:32:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80a9780 con 0x7fecdc008880 2023-11-02T15:32:56.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:56.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecdc010ea0 2023-11-02T15:32:56.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:32:56.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf7fff640 1 -- >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.175:3300/0 2023-11-02T15:32:56.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecfce9c640 1 -- >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 legacy=0x7fecdc00f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.175:6789/0 2023-11-02T15:32:56.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:56.897+0000 7fecf7fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:57.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.320+0000 7fecfce9c640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 0x7fecdc00f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51146/0 (socket says 172.21.15.174:51146) 2023-11-02T15:32:57.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.320+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 learned_addr learned my addr 172.21.15.174:0/960458067 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:32:57.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.375+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:57.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.390+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecdc010ea0 auth_method 0 2023-11-02T15:32:57.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.390+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:57.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.390+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:57.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.390+0000 7fecf7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:32:57.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.420+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:32:57.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.420+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:57.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.420+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:57.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.420+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:32:57.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.576+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 legacy=0x7fecdc00f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-02T15:32:57.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.576+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 legacy=0x7fecdc00f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-02T15:32:57.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.576+0000 7fecfce9c640 1 --1- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 0x7fecdc00f5f0 :-1 s=OPENED pgs=1 cs=1 l=1).handle_message read tag failed 2023-11-02T15:32:57.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.576+0000 7fecfce9c640 1 --1- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 0x7fecdc00f5f0 :-1 s=OPENED pgs=1 cs=1 l=1).fault on lossy channel, failing 2023-11-02T15:32:57.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:57.576+0000 7fecf77fe640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.175:6789/0 2023-11-02T15:32:58.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.053+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:58.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.053+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecdc010ea0 auth_method 0 2023-11-02T15:32:58.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.053+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:58.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.053+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:58.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.053+0000 7fecf7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:32:58.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.054+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:32:58.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.054+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:58.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.054+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:58.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.054+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:32:58.936 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.934+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:58.941 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.939+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecdc010ea0 auth_method 0 2023-11-02T15:32:58.941 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.939+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:58.942 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.939+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:58.942 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.939+0000 7fecf7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:32:58.943 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.941+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:32:58.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.941+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:58.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.941+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:58.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:58.941+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:32:59.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:32:59.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:32:59.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:32:59.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:32:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 msgr2=0x7fecdc013300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:32:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc010ea0 0x7fecdc013300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:32:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc008880 legacy=0x7fecdc00f5f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-11-02T15:32:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient: _add_conns ranks=[3,2,4,0,1] 2023-11-02T15:32:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 reap_dead start 2023-11-02T15:32:59.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:59.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc003b70 addr v2:172.21.15.174:3300/0 2023-11-02T15:32:59.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:59.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-b con 0x7fecf8144d40 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:32:59.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:32:59.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c con 0x7fecdc00fda0 addr v2:172.21.15.175:3300/0 2023-11-02T15:32:59.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:59.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecdc003b70 2023-11-02T15:32:59.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:59.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc008df0 con 0x7fecdc00fda0 2023-11-02T15:32:59.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:32:59.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0102a0 con 0x7fecf8144d40 2023-11-02T15:32:59.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.897+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:32:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.898+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.899+0000 7fecfd69d640 10 monclient(hunting): get_auth_request con 0x7fecdc00fda0 auth_method 0 2023-11-02T15:32:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.899+0000 7fecfd69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.899+0000 7fecfd69d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.899+0000 7fecfd69d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:32:59.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.900+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-02T15:32:59.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.900+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:59.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.900+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:59.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.900+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:59.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.901+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:59.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.901+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecf8144d40 auth_method 0 2023-11-02T15:32:59.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.901+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:59.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.901+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:59.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.901+0000 7fecf7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:32:59.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.902+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-02T15:32:59.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.902+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:59.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.902+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:59.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.902+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:32:59.926 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.924+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:32:59.926 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.925+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc003b70 auth_method 0 2023-11-02T15:32:59.927 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.925+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:32:59.927 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.925+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:32:59.927 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.925+0000 7fecfce9c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:32:59.928 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.926+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:32:59.928 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.926+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:32:59.928 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.926+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:32:59.928 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:32:59.926+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:33:00.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.101+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.101+0000 7fecfd69d640 10 monclient(hunting): get_auth_request con 0x7fecdc00fda0 auth_method 0 2023-11-02T15:33:00.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.101+0000 7fecfd69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.101+0000 7fecfd69d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.101+0000 7fecfd69d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.102+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.102+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.102+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.102+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:33:00.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.103+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.103+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecf8144d40 auth_method 0 2023-11-02T15:33:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.103+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.103+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.103+0000 7fecf7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.104+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.104+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.104+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.104+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:33:00.129 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.127+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.129 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.127+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc003b70 auth_method 0 2023-11-02T15:33:00.129 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.127+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.129 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.127+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.130 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.127+0000 7fecfce9c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.130 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.128+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.128+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.128+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.128+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-02T15:33:00.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.503+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.503+0000 7fecfd69d640 10 monclient(hunting): get_auth_request con 0x7fecdc00fda0 auth_method 0 2023-11-02T15:33:00.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.503+0000 7fecfd69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.503+0000 7fecfd69d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.503+0000 7fecfd69d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.504+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.504+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.504+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.504+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:33:00.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.505+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.505+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecf8144d40 auth_method 0 2023-11-02T15:33:00.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.505+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.505+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.505+0000 7fecf7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.506+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.506+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.506+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.506+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:33:00.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.529+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:00.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.529+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc003b70 auth_method 0 2023-11-02T15:33:00.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.529+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:00.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.529+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:00.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.529+0000 7fecfce9c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:00.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.530+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:00.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.530+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:00.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.530+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:00.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:00.530+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-02T15:33:01.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.306+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.306+0000 7fecfd69d640 10 monclient(hunting): get_auth_request con 0x7fecdc00fda0 auth_method 0 2023-11-02T15:33:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.306+0000 7fecfd69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.306+0000 7fecfd69d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.306+0000 7fecfd69d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.307+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.307+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.307+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.307+0000 7fecfd69d640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:33:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.307+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecf8144d40 auth_method 0 2023-11-02T15:33:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-02T15:33:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.308+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.309+0000 7fecf7fff640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:33:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.332+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.332+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc003b70 auth_method 0 2023-11-02T15:33:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.332+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.332+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.332+0000 7fecfce9c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.333+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-02T15:33:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.333+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-02T15:33:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.333+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:01.333+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-02T15:33:02.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:33:02.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:33:02.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:33:02.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:33:02.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 msgr2=0x7fecdc00b090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:33:02.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecf8144d40 0x7fecdc00b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:02.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 msgr2=0x7fecdc013290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:33:02.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.175:3300/0 conn(0x7fecdc00fda0 0x7fecdc013290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:02.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 msgr2=0x7fecdc006720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-02T15:33:02.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc003b70 0x7fecdc006720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:02.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient: _add_conns ranks=[0,1] 2023-11-02T15:33:02.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecdc010590 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:02.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fecdc0080d0 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:02.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80aa4b0 con 0x7fecdc010590 2023-11-02T15:33:02.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf81451d0 con 0x7fecdc0080d0 2023-11-02T15:33:02.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:02.897+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:05.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): tick 2023-11-02T15:33:05.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): _check_auth_tickets 2023-11-02T15:33:05.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 monclient(hunting): continuing hunt 2023-11-02T15:33:05.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): _reopen_session rank -1 2023-11-02T15:33:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc0080d0 legacy=0x7fecdc00f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.174:6789/0 conn(0x7fecdc010590 legacy=0x7fecdc013900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2023-11-02T15:33:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecfd69d640 1 -- 172.21.15.174:0/960458067 reap_dead start 2023-11-02T15:33:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc006720 0x7fecdc013900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:05.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc006720 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:05.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fecdc005b80 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:05.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecdc0080d0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:05.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc010a70 con 0x7fecdc0080d0 2023-11-02T15:33:05.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc0085e0 con 0x7fecdc005b80 2023-11-02T15:33:05.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:05.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 1 -- 172.21.15.174:0/960458067 --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80aa1d0 con 0x7fecdc006720 2023-11-02T15:33:05.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.897+0000 7fecf67fc640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:05.953 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.951+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> v2:172.21.15.174:3300/0 conn(0x7fecdc006720 0x7fecdc013900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:05.953 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.952+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc006720 auth_method 0 2023-11-02T15:33:05.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.952+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:05.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.952+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:05.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.952+0000 7fecfce9c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:05.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.981+0000 7fecfce9c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:05.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.981+0000 7fecfce9c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:05.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:05.981+0000 7fecfce9c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 10 monclient(hunting): handle_auth_done global_id 4101 payload 274 2023-11-02T15:33:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.175:6789/0 conn(0x7fecdc005b80 legacy=0x7fecdc013e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 >> v1:172.21.15.174:6789/0 conn(0x7fecdc0080d0 legacy=0x7fecdc00b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 10 monclient: _finish_hunting 0 2023-11-02T15:33:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 1 monclient: found mon.noname-a 2023-11-02T15:33:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.024+0000 7fecfce9c640 1 -- 172.21.15.174:0/960458067 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc014550 con 0x7fecdc006720 2023-11-02T15:33:06.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.045+0000 7fecfce9c640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc006720 0x7fecdc013900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fecdc008d30 tx=0x7fecec02ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faec536f77060040 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:06.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 1 -- 172.21.15.174:0/960458067 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec02d890 con 0x7fecdc006720 2023-11-02T15:33:06.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:06.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:06.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: dump: 2023-11-02T15:33:06.088 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:06.088 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:06.088 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.089 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.089 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:06.089 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:06.089 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:06.089 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:06.090 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:06.090 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:06.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:06.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:06.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:06.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 1 -- 172.21.15.174:0/960458067 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec031070 con 0x7fecdc006720 2023-11-02T15:33:06.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecff100640 5 monclient: authenticate success, global_id 4101 2023-11-02T15:33:06.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecff100640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:06.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:06.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecff100640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:06.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 1 -- 172.21.15.174:0/960458067 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec02dc50 con 0x7fecdc006720 2023-11-02T15:33:06.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecff100640 4 set_mon_vals no callback set 2023-11-02T15:33:06.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:06.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:06.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.085+0000 7fecf77fe640 10 monclient: dump: 2023-11-02T15:33:06.093 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:06.093 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:06.093 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.093 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.093 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:06.094 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:06.094 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:06.094 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:06.094 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:06.094 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:06.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 10 monclient: shutdown 2023-11-02T15:33:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 -- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc006720 msgr2=0x7fecdc013900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc006720 0x7fecdc013900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fecdc008d30 tx=0x7fecec02ccf0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 -- 172.21.15.174:0/960458067 shutdown_connections 2023-11-02T15:33:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 --2- 172.21.15.174:0/960458067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc006720 0x7fecdc013900 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 -- 172.21.15.174:0/960458067 >> 172.21.15.174:0/960458067 conn(0x7fecf8065d50 msgr2=0x7fecf8066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:06.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 -- 172.21.15.174:0/960458067 shutdown_connections 2023-11-02T15:33:06.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.086+0000 7fecff100640 1 -- 172.21.15.174:0/960458067 wait complete. 2023-11-02T15:33:06.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.087+0000 7fecff100640 10 monclient: build_initial_monmap 2023-11-02T15:33:06.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.087+0000 7fecff100640 10 monclient: monmap: 2023-11-02T15:33:06.099 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:33:06.099 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:06.099 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:06.099 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:33:06.099 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:06.100 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:06.100 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:06.100 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:06.101 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:06.101 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:06.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.087+0000 7fecff100640 1 librados: starting msgr at 2023-11-02T15:33:06.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.087+0000 7fecff100640 1 librados: starting objecter 2023-11-02T15:33:06.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 Processor -- start 2023-11-02T15:33:06.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 -- start start 2023-11-02T15:33:06.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 librados: setting wanted keys 2023-11-02T15:33:06.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 librados: calling monclient init 2023-11-02T15:33:06.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient: init 2023-11-02T15:33:06.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:06.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:33:06.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecdc005b80 0x7fecf813dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:06.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): picked mon.noname-c con 0x7fecdc005b80 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:06.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 0x7fecf80aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:06.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): picked mon.noname-b con 0x7fecdc006720 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc0080d0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80aba30 con 0x7fecdc0080d0 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf8141c10 con 0x7fecdc006720 2023-11-02T15:33:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b0a10 con 0x7fecdc005b80 2023-11-02T15:33:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.088+0000 7fecff100640 10 monclient(hunting): authenticate will time out at 1313.430478s 2023-11-02T15:33:06.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.103+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:06.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:34650/0 (socket says 172.21.15.174:34650) 2023-11-02T15:33:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 1 -- 172.21.15.174:0/1367349821 learned_addr learned my addr 172.21.15.174:0/1367349821 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 10 monclient(hunting): get_auth_request con 0x7fecdc0080d0 auth_method 0 2023-11-02T15:33:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.104+0000 7fecfd69d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:06.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.105+0000 7fecfd69d640 1 -- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 msgr2=0x7fecf80ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-11-02T15:33:06.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.105+0000 7fecfd69d640 1 -- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 msgr2=0x7fecf80ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-02T15:33:06.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.105+0000 7fecfd69d640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:33:06.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.105+0000 7fecfd69d640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-02T15:33:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.110+0000 7fecfce9c640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecdc005b80 0x7fecf813dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:06.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.119+0000 7fecfce9c640 10 monclient(hunting): get_auth_request con 0x7fecdc005b80 auth_method 0 2023-11-02T15:33:06.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.119+0000 7fecfce9c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:06.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.119+0000 7fecfce9c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:06.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.119+0000 7fecfce9c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:06.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.123+0000 7fecf7fff640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 0x7fecf80aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:06.125 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.124+0000 7fecf7fff640 10 monclient(hunting): get_auth_request con 0x7fecdc006720 auth_method 0 2023-11-02T15:33:06.125 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.124+0000 7fecf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:06.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.124+0000 7fecf7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:06.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.124+0000 7fecf7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:06.128 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.126+0000 7fecf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:06.128 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.127+0000 7fecf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:06.128 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.127+0000 7fecf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:06.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 10 monclient(hunting): handle_auth_done global_id 4102 payload 824 2023-11-02T15:33:06.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 -- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecdc005b80 msgr2=0x7fecf813dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:06.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecdc005b80 0x7fecf813dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 -- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 msgr2=0x7fecf80ab4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-02T15:33:06.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 10 monclient: _finish_hunting 0 2023-11-02T15:33:06.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 monclient: found mon.noname-b 2023-11-02T15:33:06.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:06.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:06.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.139+0000 7fecf7fff640 1 -- 172.21.15.174:0/1367349821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf80aa4b0 con 0x7fecdc006720 2023-11-02T15:33:06.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.143+0000 7fecf7fff640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 0x7fecf80aafb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:06.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.143+0000 7fecf57fa640 10 client.?.objecter ms_handle_connect 0x7fecdc006720 2023-11-02T15:33:06.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.143+0000 7fecf57fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:06.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4005a50 con 0x7fecdc006720 2023-11-02T15:33:06.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:06.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:06.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: dump: 2023-11-02T15:33:06.182 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:06.183 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:06.183 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.183 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.183 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:06.183 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:06.184 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:06.184 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:06.184 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:06.184 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:06.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: _finish_auth 0 2023-11-02T15:33:06.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: _check_auth_tickets 2023-11-02T15:33:06.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:06.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4011040 con 0x7fecdc006720 2023-11-02T15:33:06.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 5 monclient: authenticate success, global_id 4102 2023-11-02T15:33:06.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 10 monclient: _renew_subs 2023-11-02T15:33:06.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:06.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf81d98d0 con 0x7fecdc006720 2023-11-02T15:33:06.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:06.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece401b650 con 0x7fecdc006720 2023-11-02T15:33:06.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:06.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:06.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf57fa640 10 monclient: dump: 2023-11-02T15:33:06.187 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:06.189 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:06.189 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.189 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:06.189 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:06.189 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:06.190 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:06.190 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:06.190 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:06.190 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:06.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecf6ffd640 4 set_mon_vals no callback set 2023-11-02T15:33:06.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 10 client.4102.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:06.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 10 monclient: _renew_subs 2023-11-02T15:33:06.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:06.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf8142910 con 0x7fecdc006720 2023-11-02T15:33:06.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.179+0000 7fecff100640 1 librados: init done 2023-11-02T15:33:06.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.180+0000 7fecff100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:06.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.180+0000 7fecff100640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:06.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.180+0000 7fecff100640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:06.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.180+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf80aacd0 con 0x7fecdc006720 2023-11-02T15:33:06.380 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.378+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fece401b7b0 con 0x7fecdc006720 2023-11-02T15:33:06.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fece4023b30 con 0x7fecdc006720 2023-11-02T15:33:06.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 10 client.4102.objecter ms_dispatch 0x7fecf8000b90 osd_map(1..1 src has 1..1) v4 2023-11-02T15:33:06.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 3 client.4102.objecter handle_osd_map got epochs [1,1] > 0 2023-11-02T15:33:06.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 3 client.4102.objecter handle_osd_map decoding full epoch 1 2023-11-02T15:33:06.388 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 20 client.4102.objecter dump_active .. 0 homeless 2023-11-02T15:33:06.388 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fecf80aacd0 con 0x7fecdc006720 2023-11-02T15:33:06.388 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:06.388 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.385+0000 7fecf57fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:06.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.450+0000 7fecff100640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2023-11-02T15:33:06.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.450+0000 7fecff100640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-11-02T15:33:06.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.450+0000 7fecff100640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:06.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.450+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fecf8067920 con 0x7fecdc006720 2023-11-02T15:33:06.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecf57fa640 1 -- 172.21.15.174:0/1367349821 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fece401bbf0 con 0x7fecdc006720 2023-11-02T15:33:06.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecf57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-11-02T15:33:06.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecf57fa640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2023-11-02T15:33:06.695 INFO:teuthology.orchestra.run.smithi174.stderr:adjusted tunables profile to default 2023-11-02T15:33:06.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecff100640 10 librados: watch_flush enter 2023-11-02T15:33:06.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecff100640 10 librados: watch_flush exit 2023-11-02T15:33:06.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecff100640 20 client.4102.objecter shutdown clearing up homeless session... 2023-11-02T15:33:06.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.693+0000 7fecff100640 10 client.4102.objecter successfully canceled tick 2023-11-02T15:33:06.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 10 monclient: shutdown 2023-11-02T15:33:06.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:06.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 msgr2=0x7fecf80aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:06.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 0x7fecf80aafb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4009fb0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 shutdown_connections 2023-11-02T15:33:06.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecdc0080d0 0x7fecf80ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecdc006720 0x7fecf80aafb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 --2- 172.21.15.174:0/1367349821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecdc005b80 0x7fecf813dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:06.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 >> 172.21.15.174:0/1367349821 conn(0x7fecf8065d50 msgr2=0x7fecf80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:06.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 shutdown_connections 2023-11-02T15:33:06.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 -- 172.21.15.174:0/1367349821 wait complete. 2023-11-02T15:33:06.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:06.694+0000 7fecff100640 1 librados: shutdown 2023-11-02T15:33:06.709 INFO:tasks.ceph:check_enable_crimson: False 2023-11-02T15:33:06.710 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-02T15:33:06.710 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:33:06.711 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-02T15:33:06.737 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:33:06.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-02T15:33:06.770 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:06.768+0000 7f30d584b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-02T15:33:06.800 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:33:06.801 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-02T15:33:06.823 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:06.821+0000 7f30d584b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-02T15:33:06.828 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:06.826+0000 7f97c6b51040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-02T15:33:06.851 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:33:06.851 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-02T15:33:06.883 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:06.881+0000 7f97c6b51040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-02T15:33:06.918 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:33:06.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-02T15:33:06.946 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:33:06.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-02T15:33:07.013 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:33:07.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-02T15:33:07.080 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-11-02T15:33:07.081 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-02T15:33:07.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 8cc58e83-ddd7-4452-881d-72078e5d9891 0 2023-11-02T15:33:07.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 10 monclient: get_monmap_and_config 2023-11-02T15:33:07.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 10 monclient: build_initial_monmap 2023-11-02T15:33:07.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 10 monclient: monmap: 2023-11-02T15:33:07.288 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:07.288 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.288 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:07.286309+0000 2023-11-02T15:33:07.288 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:07.286309+0000 2023-11-02T15:33:07.289 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:07.289 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.289 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:07.289 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:07.289 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:07.290 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:07.290 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:07.290 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 1 Processor -- start 2023-11-02T15:33:07.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 1 -- start start 2023-11-02T15:33:07.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.285+0000 7f1838274640 10 monclient: init 2023-11-02T15:33:07.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:07.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2023-11-02T15:33:07.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f18300b4570 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:07.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18300b0ed0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:07.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 0x7f183013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): picked mon.noname-b con 0x7f1830144d40 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:07.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18300a9780 con 0x7f18300b0ed0 2023-11-02T15:33:07.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18301451d0 con 0x7f18300b4570 2023-11-02T15:33:07.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300aa1d0 con 0x7f1830144d40 2023-11-02T15:33:07.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:07.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1838274640 10 monclient(hunting): authenticate will time out at 1311.505503s 2023-11-02T15:33:07.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1836010640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f18300b4570 0x7f18300a9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37428/0 (socket says 172.21.15.175:37428) 2023-11-02T15:33:07.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.286+0000 7f1836010640 1 -- 172.21.15.175:0/1579492947 learned_addr learned my addr 172.21.15.175:0/1579492947 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-11-02T15:33:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693611097 0 0) 0x7f18301451d0 con 0x7f18300b4570 2023-11-02T15:33:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 10 monclient(hunting): _negotiate 2023-11-02T15:33:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 10 monclient(hunting): my global_id is 4105 2023-11-02T15:33:07.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.297+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f181c003520 con 0x7f18300b4570 2023-11-02T15:33:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010629803 0 0) 0x7f181c003520 con 0x7f18300b4570 2023-11-02T15:33:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 msgr2=0x7f183013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 0x7f183013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 >> v1:172.21.15.174:6789/0 conn(0x7f18300b0ed0 legacy=0x7f183013e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _finish_hunting 0 2023-11-02T15:33:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 monclient: found mon.noname-c-legacy 2023-11-02T15:33:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.175:6789/0 2023-11-02T15:33:07.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18300aa4b0 con 0x7f18300b4570 2023-11-02T15:33:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _finish_auth 0 2023-11-02T15:33:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _check_auth_tickets 2023-11-02T15:33:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 692219148 0 0) 0x7f182c001670 con 0x7f18300b4570 2023-11-02T15:33:07.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f1838274640 5 monclient: authenticate success, global_id 4105 2023-11-02T15:33:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f1838274640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2023-11-02T15:33:07.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: dump: 2023-11-02T15:33:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.311 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.312 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.313 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] but i'm connected to v1:172.21.15.175:6789/0, reconnecting 2023-11-02T15:33:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 >> v1:172.21.15.175:6789/0 conn(0x7f18300b4570 legacy=0x7f18300a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:33:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 0x7f181c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): picked mon.c con 0x7f1830144d40 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f181c0040e0 0x7f181c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): picked mon.a con 0x7f181c0040e0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:07.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f181c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): picked mon.b con 0x7f181c004e50 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300aa1d0 con 0x7f181c0040e0 2023-11-02T15:33:07.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c005670 con 0x7f181c004e50 2023-11-02T15:33:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c00bf40 con 0x7f1830144d40 2023-11-02T15:33:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:07.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f183500e640 10 monclient(hunting): _finish_auth 0 2023-11-02T15:33:07.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.304+0000 7f1838274640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:07.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.306+0000 7f1836811640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f181c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): get_auth_request con 0x7f181c004e50 auth_method 0 2023-11-02T15:33:07.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:33:07.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.308+0000 7f1836811640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-11-02T15:33:07.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 10 monclient(hunting): handle_auth_done global_id 4105 payload 293 2023-11-02T15:33:07.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 -- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 msgr2=0x7f181c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:07.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1830144d40 0x7f181c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 -- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f181c0040e0 msgr2=0x7f181c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:07.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f181c0040e0 0x7f181c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 10 monclient: _finish_hunting 0 2023-11-02T15:33:07.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 monclient: found mon.b 2023-11-02T15:33:07.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:07.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 1 -- 172.21.15.175:0/1579492947 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c003520 con 0x7f181c004e50 2023-11-02T15:33:07.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 10 monclient: _finish_auth 0 2023-11-02T15:33:07.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 10 monclient: _check_auth_tickets 2023-11-02T15:33:07.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.310+0000 7f1836811640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:07.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.312+0000 7f1836811640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f181c00b980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1828009e40 tx=0x7f181c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa739fd6f039f52e server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:07.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182802cd90 con 0x7f181c004e50 2023-11-02T15:33:07.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:07.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 10 monclient: dump: 2023-11-02T15:33:07.324 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.324 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.325 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.325 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.325 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.325 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.325 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.326 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.326 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.326 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828034440 con 0x7f181c004e50 2023-11-02T15:33:07.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:07.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f183500e640 1 -- 172.21.15.175:0/1579492947 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828034830 con 0x7f181c004e50 2023-11-02T15:33:07.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.316+0000 7f1838274640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:07.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 4 set_mon_vals no callback set 2023-11-02T15:33:07.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f183500e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f183500e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:07.328 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f183500e640 10 monclient: dump: 2023-11-02T15:33:07.328 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.328 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.328 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.328 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.329 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.329 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.329 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.329 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.329 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.330 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.334 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 10 monclient: shutdown 2023-11-02T15:33:07.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:07.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 -- 172.21.15.175:0/1579492947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 msgr2=0x7f181c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 --2- 172.21.15.175:0/1579492947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f181c00b980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1828009e40 tx=0x7f181c0057f0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1836811640 1 -- 172.21.15.175:0/1579492947 reap_dead start 2023-11-02T15:33:07.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 -- 172.21.15.175:0/1579492947 shutdown_connections 2023-11-02T15:33:07.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 -- 172.21.15.175:0/1579492947 >> 172.21.15.175:0/1579492947 conn(0x7f1830065d50 msgr2=0x7f1830066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:07.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 -- 172.21.15.175:0/1579492947 shutdown_connections 2023-11-02T15:33:07.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.317+0000 7f1838274640 1 -- 172.21.15.175:0/1579492947 wait complete. 2023-11-02T15:33:07.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 10 monclient: build_initial_monmap 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 10 monclient: monmap: 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:07.337 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:07.338 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:07.338 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 librados: starting msgr at 2023-11-02T15:33:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 librados: starting objecter 2023-11-02T15:33:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 Processor -- start 2023-11-02T15:33:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 -- start start 2023-11-02T15:33:07.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 librados: setting wanted keys 2023-11-02T15:33:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 1 librados: calling monclient init 2023-11-02T15:33:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.318+0000 7f1838274640 10 monclient: init 2023-11-02T15:33:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:07.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): picked mon.noname-b con 0x7f181c004e50 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1830142730 0x7f18300aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): picked mon.noname-a con 0x7f1830142730 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18300acfb0 0x7f18301d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): picked mon.noname-c con 0x7f18300acfb0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301d9a70 con 0x7f1830142730 2023-11-02T15:33:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301da4d0 con 0x7f181c004e50 2023-11-02T15:33:07.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18301da640 con 0x7f18300acfb0 2023-11-02T15:33:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1838274640 10 monclient(hunting): authenticate will time out at 1311.538653s 2023-11-02T15:33:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.319+0000 7f1836010640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55012/0 (socket says 172.21.15.175:55012) 2023-11-02T15:33:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 1 -- 172.21.15.175:0/2935851144 learned_addr learned my addr 172.21.15.175:0/2935851144 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): get_auth_request con 0x7f181c004e50 auth_method 0 2023-11-02T15:33:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.320+0000 7f1836010640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:07.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 10 monclient(hunting): handle_auth_done global_id 4108 payload 824 2023-11-02T15:33:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 -- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18300acfb0 msgr2=0x7f18301d9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18300acfb0 0x7f18301d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.346 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 -- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1830142730 msgr2=0x7f18300aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1830142730 0x7f18300aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 10 monclient: _finish_hunting 0 2023-11-02T15:33:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 monclient: found mon.noname-b 2023-11-02T15:33:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:07.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.323+0000 7f1836010640 1 -- 172.21.15.175:0/2935851144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18301da800 con 0x7f181c004e50 2023-11-02T15:33:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.326+0000 7f1836010640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c680 tx=0x7f182c00cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.326+0000 7f1826ffd640 10 client.?.objecter ms_handle_connect 0x7f181c004e50 2023-11-02T15:33:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.326+0000 7f1826ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:07.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0072d0 con 0x7f181c004e50 2023-11-02T15:33:07.349 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.349 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:07.349 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: dump: 2023-11-02T15:33:07.349 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.349 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.350 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.351 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.351 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c007430 con 0x7f181c004e50 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 5 monclient: authenticate success, global_id 4108 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 10 monclient: _renew_subs 2023-11-02T15:33:07.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18301394d0 con 0x7f181c004e50 2023-11-02T15:33:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0047e0 con 0x7f181c004e50 2023-11-02T15:33:07.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1826ffd640 10 monclient: dump: 2023-11-02T15:33:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.354 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.355 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.356 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f183480d640 4 set_mon_vals no callback set 2023-11-02T15:33:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 10 client.4108.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:07.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 10 monclient: _renew_subs 2023-11-02T15:33:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18301399e0 con 0x7f181c004e50 2023-11-02T15:33:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.330+0000 7f1838274640 1 librados: init done 2023-11-02T15:33:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.331+0000 7f1838274640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.331+0000 7f1838274640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.331+0000 7f1838274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.331+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830067920 con 0x7f181c004e50 2023-11-02T15:33:07.385 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.383+0000 7f30d584b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-02T15:33:07.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f182c004c40 con 0x7f181c004e50 2023-11-02T15:33:07.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f182c012400 con 0x7f181c004e50 2023-11-02T15:33:07.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 10 client.4108.objecter ms_dispatch 0x7f1830000b90 osd_map(2..2 src has 1..2) v4 2023-11-02T15:33:07.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 3 client.4108.objecter handle_osd_map got epochs [2,2] > 0 2023-11-02T15:33:07.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 3 client.4108.objecter handle_osd_map decoding full epoch 2 2023-11-02T15:33:07.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 20 client.4108.objecter dump_active .. 0 homeless 2023-11-02T15:33:07.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.428+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f182c018070 con 0x7f181c004e50 2023-11-02T15:33:07.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.429+0000 7f1826ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.429+0000 7f1826ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:07.437 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.436+0000 7f30d584b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-02T15:33:07.460 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.458+0000 7f97c6b51040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-02T15:33:07.496 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.495+0000 7f30d584b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-02T15:33:07.501 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.499+0000 7f1838274640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8cc58e83-ddd7-4452-881d-72078e5d9891", "id": 0}] 2023-11-02T15:33:07.502 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.499+0000 7f1838274640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8cc58e83-ddd7-4452-881d-72078e5d9891", "id": 0}] 2023-11-02T15:33:07.502 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.499+0000 7f1838274640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:07.502 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.499+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8cc58e83-ddd7-4452-881d-72078e5d9891", "id": 0} v 0) v1 -- 0x7f18300b52d0 con 0x7f181c004e50 2023-11-02T15:33:07.513 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.512+0000 7f97c6b51040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-02T15:33:07.551 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.549+0000 7f30d584b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-02T15:33:07.557 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.556+0000 7f97c6b51040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-02T15:33:07.597 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.596+0000 7f30d584b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-02T15:33:07.611 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.610+0000 7f97c6b51040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-02T15:33:07.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.650+0000 7f1826ffd640 1 -- 172.21.15.175:0/2935851144 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8cc58e83-ddd7-4452-881d-72078e5d9891", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f182c00a6b0 con 0x7f181c004e50 2023-11-02T15:33:07.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.650+0000 7f1826ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8cc58e83-ddd7-4452-881d-72078e5d9891", "id": 0}] 2023-11-02T15:33:07.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.650+0000 7f1826ffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:07.653 INFO:teuthology.orchestra.run.smithi175.stdout:0 2023-11-02T15:33:07.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 10 librados: watch_flush enter 2023-11-02T15:33:07.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 10 librados: watch_flush exit 2023-11-02T15:33:07.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 20 client.4108.objecter shutdown clearing up homeless session... 2023-11-02T15:33:07.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 10 client.4108.objecter successfully canceled tick 2023-11-02T15:33:07.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 10 monclient: shutdown 2023-11-02T15:33:07.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:07.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 msgr2=0x7f18300abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c680 tx=0x7f182c00cb40 comp rx=0 tx=0).stop 2023-11-02T15:33:07.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 shutdown_connections 2023-11-02T15:33:07.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18300acfb0 0x7f18301d9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1830142730 0x7f18300aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 --2- 172.21.15.175:0/2935851144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c004e50 0x7f18300abc30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 >> 172.21.15.175:0/2935851144 conn(0x7f1830065d50 msgr2=0x7f1830073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:07.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 shutdown_connections 2023-11-02T15:33:07.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 -- 172.21.15.175:0/2935851144 wait complete. 2023-11-02T15:33:07.658 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.651+0000 7f1838274640 1 librados: shutdown 2023-11-02T15:33:07.658 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.653+0000 7f97c6b51040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-02T15:33:07.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 8be55432-eb0e-4795-8de9-3831cc022e9c 1 2023-11-02T15:33:07.731 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.729+0000 7f30d584b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-02T15:33:07.761 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f97c6b51040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-02T15:33:07.762 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 10 monclient: get_monmap_and_config 2023-11-02T15:33:07.762 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 10 monclient: build_initial_monmap 2023-11-02T15:33:07.763 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 10 monclient: monmap: 2023-11-02T15:33:07.763 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:07.763 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.763 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:07.760476+0000 2023-11-02T15:33:07.763 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:07.760476+0000 2023-11-02T15:33:07.764 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:07.764 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.764 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:07.764 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:07.764 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:07.765 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:07.765 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:07.765 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 1 Processor -- start 2023-11-02T15:33:07.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 1 -- start start 2023-11-02T15:33:07.766 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.759+0000 7f8821d73640 10 monclient: init 2023-11-02T15:33:07.766 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:07.766 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient: _add_conns ranks=[0,2,1,4,3] 2023-11-02T15:33:07.766 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f881c0b4570 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:07.766 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 0x7f881c144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-b con 0x7f881c0b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:07.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f881c13e520 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:07.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881c0a9780 con 0x7f881c0b4570 2023-11-02T15:33:07.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881c0aa1d0 con 0x7f881c13e520 2023-11-02T15:33:07.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0b4950 con 0x7f881c0b0ed0 2023-11-02T15:33:07.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:07.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.760+0000 7f8821d73640 10 monclient(hunting): authenticate will time out at 1311.979789s 2023-11-02T15:33:07.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881bfff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f881c13e520 0x7f881c13e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37432/0 (socket says 172.21.15.175:37432) 2023-11-02T15:33:07.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881bfff640 1 -- 172.21.15.175:0/3591214548 learned_addr learned my addr 172.21.15.175:0/3591214548 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-11-02T15:33:07.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881affd640 1 --2- 172.21.15.175:0/3591214548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 0x7f881c144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881affd640 10 monclient(hunting): get_auth_request con 0x7f881c0b0ed0 auth_method 0 2023-11-02T15:33:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.761+0000 7f881affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.762+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.762+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.762+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 1 -- 172.21.15.175:0/3591214548 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432788840 0 0) 0x7f881c0aa1d0 con 0x7f881c13e520 2023-11-02T15:33:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 10 monclient(hunting): _negotiate 2023-11-02T15:33:07.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 10 monclient(hunting): my global_id is 4111 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881a7fc640 1 -- 172.21.15.175:0/3591214548 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8804003520 con 0x7f881c13e520 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 10 monclient(hunting): handle_auth_done global_id 4103 payload 274 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 1 -- 172.21.15.175:0/3591214548 >> v1:172.21.15.175:6789/0 conn(0x7f881c13e520 legacy=0x7f881c13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 1 -- 172.21.15.175:0/3591214548 >> v1:172.21.15.174:6789/0 conn(0x7f881c0b4570 legacy=0x7f881c0a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 10 monclient: _finish_hunting 0 2023-11-02T15:33:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 1 monclient: found mon.noname-b 2023-11-02T15:33:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:33:07.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.764+0000 7f881affd640 1 -- 172.21.15.175:0/3591214548 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0aa4b0 con 0x7f881c0b0ed0 2023-11-02T15:33:07.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.766+0000 7f881affd640 1 --2- 172.21.15.175:0/3591214548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 0x7f881c144d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88100097c0 tx=0x7f881002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b5cd767ab1c018 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:07.779 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 1 -- 172.21.15.175:0/3591214548 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810031760 con 0x7f881c0b0ed0 2023-11-02T15:33:07.779 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.779 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:07.780 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: dump: 2023-11-02T15:33:07.780 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.780 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.780 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.780 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.781 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.781 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.781 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.781 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.781 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.782 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: _finish_auth 0 2023-11-02T15:33:07.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:33:07.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:07.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 1 -- 172.21.15.175:0/3591214548 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88100318c0 con 0x7f881c0b0ed0 2023-11-02T15:33:07.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f8821d73640 5 monclient: authenticate success, global_id 4103 2023-11-02T15:33:07.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f8821d73640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:07.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:07.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f8821d73640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:07.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f8821d73640 4 set_mon_vals no callback set 2023-11-02T15:33:07.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 1 -- 172.21.15.175:0/3591214548 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810031c50 con 0x7f881c0b0ed0 2023-11-02T15:33:07.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:07.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.771+0000 7f881a7fc640 10 monclient: dump: 2023-11-02T15:33:07.784 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.785 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.785 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.785 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.785 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.786 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.786 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.786 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.786 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.786 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 10 monclient: shutdown 2023-11-02T15:33:07.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:07.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 -- 172.21.15.175:0/3591214548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 msgr2=0x7f881c144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 --2- 172.21.15.175:0/3591214548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 0x7f881c144d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f88100097c0 tx=0x7f881002b800 comp rx=0 tx=0).stop 2023-11-02T15:33:07.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 -- 172.21.15.175:0/3591214548 shutdown_connections 2023-11-02T15:33:07.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 --2- 172.21.15.175:0/3591214548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c0b0ed0 0x7f881c144d40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 -- 172.21.15.175:0/3591214548 >> 172.21.15.175:0/3591214548 conn(0x7f881c065d50 msgr2=0x7f881c066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:07.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 -- 172.21.15.175:0/3591214548 shutdown_connections 2023-11-02T15:33:07.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.772+0000 7f8821d73640 1 -- 172.21.15.175:0/3591214548 wait complete. 2023-11-02T15:33:07.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 10 monclient: build_initial_monmap 2023-11-02T15:33:07.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 10 monclient: monmap: 2023-11-02T15:33:07.789 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:07.789 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:07.790 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:07.791 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:07.791 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 librados: starting msgr at 2023-11-02T15:33:07.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 librados: starting objecter 2023-11-02T15:33:07.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 Processor -- start 2023-11-02T15:33:07.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 -- start start 2023-11-02T15:33:07.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 librados: setting wanted keys 2023-11-02T15:33:07.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 1 librados: calling monclient init 2023-11-02T15:33:07.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.773+0000 7f8821d73640 10 monclient: init 2023-11-02T15:33:07.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:07.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T15:33:07.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f881c0b0ed0 0x7f881c0ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-a con 0x7f881c0b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:07.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 0x7f881c1d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-b con 0x7f881c0b4570 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:07.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 0x7f881c1da240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:07.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): picked mon.noname-c con 0x7f881c13e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:07.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0acf90 con 0x7f881c0b0ed0 2023-11-02T15:33:07.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.795 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c142440 con 0x7f881c0b4570 2023-11-02T15:33:07.795 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:07.795 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c1425b0 con 0x7f881c13e520 2023-11-02T15:33:07.795 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:07.795 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f8821d73640 10 monclient(hunting): authenticate will time out at 1311.993443s 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f881affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 0x7f881c1d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f881bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 0x7f881c1da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.774+0000 7f881b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f881c0b0ed0 0x7f881c0ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 0x7f881c1d9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55016/0 (socket says 172.21.15.175:55016) 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 1 -- 172.21.15.175:0/2842824611 learned_addr learned my addr 172.21.15.175:0/2842824611 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:07.796 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): get_auth_request con 0x7f881c0b4570 auth_method 0 2023-11-02T15:33:07.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): get_auth_request con 0x7f881c13e520 auth_method 0 2023-11-02T15:33:07.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): get_auth_request con 0x7f881c0b0ed0 auth_method 0 2023-11-02T15:33:07.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:07.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:07.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:07.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.801 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:07.801 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:07.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:07.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:07.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.775+0000 7f881b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:07.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 10 monclient(hunting): handle_auth_done global_id 4106 payload 824 2023-11-02T15:33:07.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 -- 172.21.15.175:0/2842824611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 msgr2=0x7f881c1d9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 0x7f881c1d9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 -- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f881c0b0ed0 msgr2=0x7f881c0ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:07.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f881c0b0ed0 0x7f881c0ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:07.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 10 monclient: _finish_hunting 0 2023-11-02T15:33:07.806 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 monclient: found mon.noname-c 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.778+0000 7f881bfff640 1 -- 172.21.15.175:0/2842824611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100093a0 con 0x7f881c13e520 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.780+0000 7f881bfff640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 0x7f881c1da240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f881400caa0 tx=0x7f881400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.780+0000 7f8803fff640 10 client.?.objecter ms_handle_connect 0x7f881c13e520 2023-11-02T15:33:07.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.780+0000 7f8803fff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:07.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881400ab40 con 0x7f881c13e520 2023-11-02T15:33:07.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:07.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: dump: 2023-11-02T15:33:07.808 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.809 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.809 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.809 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.809 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.809 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.810 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.810 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.810 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.810 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: _finish_auth 0 2023-11-02T15:33:07.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: _check_auth_tickets 2023-11-02T15:33:07.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:07.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881400aca0 con 0x7f881c13e520 2023-11-02T15:33:07.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:07.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814012600 con 0x7f881c13e520 2023-11-02T15:33:07.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 5 monclient: authenticate success, global_id 4106 2023-11-02T15:33:07.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 10 monclient: _renew_subs 2023-11-02T15:33:07.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:07.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c0ad4b0 con 0x7f881c13e520 2023-11-02T15:33:07.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8803fff640 10 monclient: dump: 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.813 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:07.814 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:07.814 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:07.814 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:07.814 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:07.814 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:07.815 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:07.815 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8819ffb640 4 set_mon_vals no callback set 2023-11-02T15:33:07.815 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 10 client.4106.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:07.815 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 10 monclient: _renew_subs 2023-11-02T15:33:07.815 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:07.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c0acce0 con 0x7f881c13e520 2023-11-02T15:33:07.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.784+0000 7f8821d73640 1 librados: init done 2023-11-02T15:33:07.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.785+0000 7f8821d73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.785+0000 7f8821d73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.785+0000 7f8821d73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.785+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c0ad6a0 con 0x7f881c13e520 2023-11-02T15:33:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8814012a60 con 0x7f881c13e520 2023-11-02T15:33:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f881401caf0 con 0x7f881c13e520 2023-11-02T15:33:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 10 client.4106.objecter ms_dispatch 0x7f881c000b90 osd_map(3..3 src has 1..3) v4 2023-11-02T15:33:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 3 client.4106.objecter handle_osd_map got epochs [3,3] > 0 2023-11-02T15:33:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 3 client.4106.objecter handle_osd_map decoding full epoch 3 2023-11-02T15:33:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.788+0000 7f8803fff640 20 client.4106.objecter dump_active .. 0 homeless 2023-11-02T15:33:07.818 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.803+0000 7f30d584b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-02T15:33:07.834 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.832+0000 7f97c6b51040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-02T15:33:07.850 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.849+0000 7f30d584b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-02T15:33:07.880 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.878+0000 7f97c6b51040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-02T15:33:07.904 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.903+0000 7f30d584b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-02T15:33:07.932 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.930+0000 7f97c6b51040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-02T15:33:07.952 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:07.951+0000 7f30d584b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-02T15:33:07.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.963+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f8814021060 con 0x7f881c13e520 2023-11-02T15:33:07.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.963+0000 7f8803fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:07.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:07.963+0000 7f8803fff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:07.979 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:07.977+0000 7f97c6b51040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-02T15:33:08.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.034+0000 7f8821d73640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8be55432-eb0e-4795-8de9-3831cc022e9c", "id": 1}] 2023-11-02T15:33:08.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.034+0000 7f8821d73640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8be55432-eb0e-4795-8de9-3831cc022e9c", "id": 1}] 2023-11-02T15:33:08.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.034+0000 7f8821d73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:08.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.034+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8be55432-eb0e-4795-8de9-3831cc022e9c", "id": 1} v 0) v1 -- 0x7f881c067920 con 0x7f881c13e520 2023-11-02T15:33:08.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8803fff640 1 -- 172.21.15.175:0/2842824611 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8be55432-eb0e-4795-8de9-3831cc022e9c", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8814017070 con 0x7f881c13e520 2023-11-02T15:33:08.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8803fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8be55432-eb0e-4795-8de9-3831cc022e9c", "id": 1}] 2023-11-02T15:33:08.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8803fff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:08.162 INFO:teuthology.orchestra.run.smithi175.stdout:1 2023-11-02T15:33:08.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 10 librados: watch_flush enter 2023-11-02T15:33:08.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 10 librados: watch_flush exit 2023-11-02T15:33:08.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 20 client.4106.objecter shutdown clearing up homeless session... 2023-11-02T15:33:08.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 10 client.4106.objecter successfully canceled tick 2023-11-02T15:33:08.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 10 monclient: shutdown 2023-11-02T15:33:08.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:08.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 msgr2=0x7f881c1da240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.160+0000 7f8821d73640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 0x7f881c1da240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f881400caa0 tx=0x7f881400cea0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 shutdown_connections 2023-11-02T15:33:08.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f881c13e520 0x7f881c1da240 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c0b4570 0x7f881c1d9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 --2- 172.21.15.175:0/2842824611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f881c0b0ed0 0x7f881c0ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 >> 172.21.15.175:0/2842824611 conn(0x7f881c065d50 msgr2=0x7f881c13e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 shutdown_connections 2023-11-02T15:33:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 -- 172.21.15.175:0/2842824611 wait complete. 2023-11-02T15:33:08.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.161+0000 7f8821d73640 1 librados: shutdown 2023-11-02T15:33:08.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 396e4073-1411-4e20-b491-1f63a71809a3 2 2023-11-02T15:33:08.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.269+0000 7f54294b6640 10 monclient: get_monmap_and_config 2023-11-02T15:33:08.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.269+0000 7f54294b6640 10 monclient: build_initial_monmap 2023-11-02T15:33:08.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.269+0000 7f54294b6640 10 monclient: monmap: 2023-11-02T15:33:08.271 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:08.271 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.271 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:08.269775+0000 2023-11-02T15:33:08.271 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:08.269775+0000 2023-11-02T15:33:08.272 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:08.272 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.272 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:08.272 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:08.272 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:08.273 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:08.273 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:08.273 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.275 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 Processor -- start 2023-11-02T15:33:08.276 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 -- start start 2023-11-02T15:33:08.276 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient: init 2023-11-02T15:33:08.276 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:08.276 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient: _add_conns ranks=[0,3,1,4,2] 2023-11-02T15:33:08.276 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f54240b7be0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:08.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f54240a9420 0x7f54240a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-a con 0x7f54240a9420 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:08.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f542413e930 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:08.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240aa260 con 0x7f54240b7be0 2023-11-02T15:33:08.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240aa4b0 con 0x7f542413e930 2023-11-02T15:33:08.278 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.278 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54240b7fc0 con 0x7f54240a9420 2023-11-02T15:33:08.278 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:08.278 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.270+0000 7f54294b6640 10 monclient(hunting): authenticate will time out at 1312.489057s 2023-11-02T15:33:08.279 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.271+0000 7f54237fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f542413e930 0x7f542413eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37446/0 (socket says 172.21.15.175:37446) 2023-11-02T15:33:08.279 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.271+0000 7f54237fe640 1 -- 172.21.15.175:0/830321790 learned_addr learned my addr 172.21.15.175:0/830321790 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-11-02T15:33:08.279 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.271+0000 7f541bfff640 1 --2- 172.21.15.175:0/830321790 >> v2:172.21.15.174:3300/0 conn(0x7f54240a9420 0x7f54240a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.279 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): get_auth_request con 0x7f54240a9420 auth_method 0 2023-11-02T15:33:08.279 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.272+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 1 -- 172.21.15.175:0/830321790 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946347692 0 0) 0x7f54240aa4b0 con 0x7f542413e930 2023-11-02T15:33:08.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 10 monclient(hunting): _negotiate 2023-11-02T15:33:08.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 10 monclient(hunting): my global_id is 4117 2023-11-02T15:33:08.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f54227fc640 1 -- 172.21.15.175:0/830321790 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5410003520 con 0x7f542413e930 2023-11-02T15:33:08.283 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 10 monclient(hunting): handle_auth_done global_id 4119 payload 274 2023-11-02T15:33:08.283 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 1 -- 172.21.15.175:0/830321790 >> v1:172.21.15.175:6789/0 conn(0x7f542413e930 legacy=0x7f542413eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.283 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 1 -- 172.21.15.175:0/830321790 >> v1:172.21.15.174:6789/0 conn(0x7f54240b7be0 legacy=0x7f5424144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.283 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 10 monclient: _finish_hunting 0 2023-11-02T15:33:08.283 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 1 monclient: found mon.noname-a 2023-11-02T15:33:08.284 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:08.284 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:08.284 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.274+0000 7f541bfff640 1 -- 172.21.15.175:0/830321790 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240aa790 con 0x7f54240a9420 2023-11-02T15:33:08.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.279+0000 7f541bfff640 1 --2- 172.21.15.175:0/830321790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240a9420 0x7f54240a8870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c10ef08a738cd2c server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:08.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 1 -- 172.21.15.175:0/830321790 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c030610 con 0x7f54240a9420 2023-11-02T15:33:08.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:08.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: dump: 2023-11-02T15:33:08.289 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.289 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.289 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.289 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.289 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.290 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.290 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.290 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.290 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.290 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: _finish_auth 0 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: _check_auth_tickets 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 1 -- 172.21.15.175:0/830321790 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c030770 con 0x7f54240a9420 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:08.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 1 -- 172.21.15.175:0/830321790 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c030b00 con 0x7f54240a9420 2023-11-02T15:33:08.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54294b6640 5 monclient: authenticate success, global_id 4119 2023-11-02T15:33:08.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54294b6640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:08.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54294b6640 4 set_mon_vals no callback set 2023-11-02T15:33:08.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:08.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.282+0000 7f54227fc640 10 monclient: dump: 2023-11-02T15:33:08.293 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.293 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.294 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.294 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.294 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.294 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.294 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.295 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.295 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.295 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 10 monclient: shutdown 2023-11-02T15:33:08.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:08.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 -- 172.21.15.175:0/830321790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240a9420 msgr2=0x7f54240a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 --2- 172.21.15.175:0/830321790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240a9420 0x7f54240a8870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02b960 comp rx=0 tx=0).stop 2023-11-02T15:33:08.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 -- 172.21.15.175:0/830321790 shutdown_connections 2023-11-02T15:33:08.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 --2- 172.21.15.175:0/830321790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240a9420 0x7f54240a8870 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 -- 172.21.15.175:0/830321790 >> 172.21.15.175:0/830321790 conn(0x7f542406c510 msgr2=0x7f54240b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:08.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 -- 172.21.15.175:0/830321790 shutdown_connections 2023-11-02T15:33:08.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.283+0000 7f54294b6640 1 -- 172.21.15.175:0/830321790 wait complete. 2023-11-02T15:33:08.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 10 monclient: build_initial_monmap 2023-11-02T15:33:08.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 10 monclient: monmap: 2023-11-02T15:33:08.297 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:08.298 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:08.298 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:08.298 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:08.298 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:08.298 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.299 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:08.299 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:08.299 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:08.299 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 librados: starting msgr at 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 librados: starting objecter 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 Processor -- start 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 -- start start 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 librados: setting wanted keys 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 1 librados: calling monclient init 2023-11-02T15:33:08.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.284+0000 7f54294b6640 10 monclient: init 2023-11-02T15:33:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-b con 0x7f54240a9420 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 0x7f542413a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-a con 0x7f54240b7be0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:08.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f542413e930 0x7f54240ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): picked mon.noname-c con 0x7f542413e930 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:08.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424139880 con 0x7f54240b7be0 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542413a550 con 0x7f54240a9420 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542413b010 con 0x7f542413e930 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:08.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f54294b6640 10 monclient(hunting): authenticate will time out at 1312.503395s 2023-11-02T15:33:08.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f5422ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.285+0000 7f541bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 0x7f542413a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55032/0 (socket says 172.21.15.175:55032) 2023-11-02T15:33:08.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 1 -- 172.21.15.175:0/4086449641 learned_addr learned my addr 172.21.15.175:0/4086449641 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:08.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): get_auth_request con 0x7f54240a9420 auth_method 0 2023-11-02T15:33:08.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f54237fe640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f542413e930 0x7f54240ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): get_auth_request con 0x7f54240b7be0 auth_method 0 2023-11-02T15:33:08.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f54237fe640 10 monclient(hunting): get_auth_request con 0x7f542413e930 auth_method 0 2023-11-02T15:33:08.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f54237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f54237fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f54237fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f5422ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.286+0000 7f541bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.287+0000 7f54237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.287+0000 7f54237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.287+0000 7f54237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2023-11-02T15:33:08.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 -- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f542413e930 msgr2=0x7f54240ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f542413e930 0x7f54240ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 -- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 msgr2=0x7f542413a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 0x7f542413a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 10 monclient: _finish_hunting 0 2023-11-02T15:33:08.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 monclient: found mon.noname-b 2023-11-02T15:33:08.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:08.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f5422ffd640 1 -- 172.21.15.175:0/4086449641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c009400 con 0x7f54240a9420 2023-11-02T15:33:08.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.289+0000 7f541bfff640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 0x7f542413a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T15:33:08.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.292+0000 7f5422ffd640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f541400c450 tx=0x7f541400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:08.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.292+0000 7f541b7fe640 10 client.?.objecter ms_handle_connect 0x7f54240a9420 2023-11-02T15:33:08.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.292+0000 7f541b7fe640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:08.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140052d0 con 0x7f54240a9420 2023-11-02T15:33:08.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:08.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: dump: 2023-11-02T15:33:08.314 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.315 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.316 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.316 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.316 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:08.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:08.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:08.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5414005430 con 0x7f54240a9420 2023-11-02T15:33:08.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 5 monclient: authenticate success, global_id 4120 2023-11-02T15:33:08.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:08.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414012870 con 0x7f54240a9420 2023-11-02T15:33:08.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 10 monclient: _renew_subs 2023-11-02T15:33:08.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424138330 con 0x7f54240a9420 2023-11-02T15:33:08.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f5421ffb640 4 set_mon_vals no callback set 2023-11-02T15:33:08.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f541b7fe640 10 monclient: dump: 2023-11-02T15:33:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.319 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.320 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 10 client.4120.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 10 monclient: _renew_subs 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54240abac0 con 0x7f54240a9420 2023-11-02T15:33:08.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.295+0000 7f54294b6640 1 librados: init done 2023-11-02T15:33:08.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.296+0000 7f54294b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.296+0000 7f54294b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.296+0000 7f54294b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.296+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54241428e0 con 0x7f54240a9420 2023-11-02T15:33:08.348 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.346+0000 7f30d584b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-02T15:33:08.371 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.370+0000 7f97c6b51040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-02T15:33:08.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5414012cd0 con 0x7f54240a9420 2023-11-02T15:33:08.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5414006d60 con 0x7f54240a9420 2023-11-02T15:33:08.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 10 client.4120.objecter ms_dispatch 0x7f5424000b90 osd_map(4..4 src has 1..4) v4 2023-11-02T15:33:08.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 3 client.4120.objecter handle_osd_map got epochs [4,4] > 0 2023-11-02T15:33:08.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 3 client.4120.objecter handle_osd_map decoding full epoch 4 2023-11-02T15:33:08.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 20 client.4120.objecter dump_active .. 0 homeless 2023-11-02T15:33:08.398 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f54140035f0 con 0x7f54240a9420 2023-11-02T15:33:08.398 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.398 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.394+0000 7f541b7fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:08.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.478+0000 7f54294b6640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "396e4073-1411-4e20-b491-1f63a71809a3", "id": 2}] 2023-11-02T15:33:08.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.478+0000 7f54294b6640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "396e4073-1411-4e20-b491-1f63a71809a3", "id": 2}] 2023-11-02T15:33:08.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.478+0000 7f54294b6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.478+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "396e4073-1411-4e20-b491-1f63a71809a3", "id": 2} v 0) v1 -- 0x7f5424139c40 con 0x7f54240a9420 2023-11-02T15:33:08.503 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.501+0000 7f30d584b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-02T15:33:08.509 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.509+0000 7f97c6b51040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-02T15:33:08.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.570+0000 7f541b7fe640 1 -- 172.21.15.175:0/4086449641 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "396e4073-1411-4e20-b491-1f63a71809a3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5414006a30 con 0x7f54240a9420 2023-11-02T15:33:08.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.570+0000 7f541b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "396e4073-1411-4e20-b491-1f63a71809a3", "id": 2}] 2023-11-02T15:33:08.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.570+0000 7f541b7fe640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:08.572 INFO:teuthology.orchestra.run.smithi175.stdout:2 2023-11-02T15:33:08.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 10 librados: watch_flush enter 2023-11-02T15:33:08.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 10 librados: watch_flush exit 2023-11-02T15:33:08.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 20 client.4120.objecter shutdown clearing up homeless session... 2023-11-02T15:33:08.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 10 client.4120.objecter successfully canceled tick 2023-11-02T15:33:08.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 10 monclient: shutdown 2023-11-02T15:33:08.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:08.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 msgr2=0x7f5424141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f541400c450 tx=0x7f541400c910 comp rx=0 tx=0).stop 2023-11-02T15:33:08.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 shutdown_connections 2023-11-02T15:33:08.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f542413e930 0x7f54240ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54240b7be0 0x7f542413a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 --2- 172.21.15.175:0/4086449641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54240a9420 0x7f5424141e00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 >> 172.21.15.175:0/4086449641 conn(0x7f542406c510 msgr2=0x7f542413b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 shutdown_connections 2023-11-02T15:33:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 -- 172.21.15.175:0/4086449641 wait complete. 2023-11-02T15:33:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.571+0000 7f54294b6640 1 librados: shutdown 2023-11-02T15:33:08.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 751b6bb7-19d9-47ab-a641-5559f3e6d131 3 2023-11-02T15:33:08.622 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.621+0000 7f97c6b51040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-02T15:33:08.643 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.641+0000 7f30d584b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-02T15:33:08.669 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.668+0000 7f97c6b51040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-02T15:33:08.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.681+0000 7f87226db640 10 monclient: get_monmap_and_config 2023-11-02T15:33:08.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.681+0000 7f87226db640 10 monclient: build_initial_monmap 2023-11-02T15:33:08.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.681+0000 7f87226db640 10 monclient: monmap: 2023-11-02T15:33:08.683 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:08.683 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.683 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:08.681833+0000 2023-11-02T15:33:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:08.681833+0000 2023-11-02T15:33:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:08.685 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:08.685 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:08.685 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:08.685 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.687 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 Processor -- start 2023-11-02T15:33:08.687 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 -- start start 2023-11-02T15:33:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient: init 2023-11-02T15:33:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient: _add_conns ranks=[0,1,4,2,3] 2023-11-02T15:33:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f871c0b4570 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f871c0b0ed0 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:08.689 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f871c144d40 0x7f871c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.689 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): picked mon.noname-c con 0x7f871c144d40 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:08.689 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0a9780 con 0x7f871c0b4570 2023-11-02T15:33:08.689 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c1451d0 con 0x7f871c0b0ed0 2023-11-02T15:33:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c0aa1d0 con 0x7f871c144d40 2023-11-02T15:33:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.682+0000 7f87226db640 10 monclient(hunting): authenticate will time out at 1312.901048s 2023-11-02T15:33:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f8720c78640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f871c144d40 0x7f871c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f871bfff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f871c0b4570 0x7f871c0a9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.175:43188/0 (socket says 172.21.15.175:43188) 2023-11-02T15:33:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f871bfff640 1 -- 172.21.15.175:0/5028333 learned_addr learned my addr 172.21.15.175:0/5028333 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-11-02T15:33:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f8720c78640 10 monclient(hunting): get_auth_request con 0x7f871c144d40 auth_method 0 2023-11-02T15:33:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f8720c78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f8720c78640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.683+0000 7f8720c78640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.684+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.684+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.684+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 10 monclient(hunting): handle_auth_done global_id 4123 payload 274 2023-11-02T15:33:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 1 -- 172.21.15.175:0/5028333 >> v1:172.21.15.175:6789/0 conn(0x7f871c0b0ed0 legacy=0x7f871c13e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 1 -- 172.21.15.175:0/5028333 >> v1:172.21.15.174:6789/0 conn(0x7f871c0b4570 legacy=0x7f871c0a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 10 monclient: _finish_hunting 0 2023-11-02T15:33:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 1 monclient: found mon.noname-c 2023-11-02T15:33:08.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:08.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.686+0000 7f8720c78640 1 -- 172.21.15.175:0/5028333 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c0aa4b0 con 0x7f871c144d40 2023-11-02T15:33:08.695 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.688+0000 7f30d584b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-02T15:33:08.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.688+0000 7f8720c78640 1 --2- 172.21.15.175:0/5028333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c13ec30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8714009dc0 tx=0x7f871402bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde23704c58e9879 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:08.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 1 -- 172.21.15.175:0/5028333 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871402cb20 con 0x7f871c144d40 2023-11-02T15:33:08.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.703 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:08.703 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: dump: 2023-11-02T15:33:08.703 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.704 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.704 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.704 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.706 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.706 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.706 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: _finish_auth 0 2023-11-02T15:33:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f87226db640 5 monclient: authenticate success, global_id 4123 2023-11-02T15:33:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f87226db640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:08.708 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 1 -- 172.21.15.175:0/5028333 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871402cc80 con 0x7f871c144d40 2023-11-02T15:33:08.708 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:08.708 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f87226db640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:08.709 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 1 -- 172.21.15.175:0/5028333 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714034440 con 0x7f871c144d40 2023-11-02T15:33:08.709 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.709 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f87226db640 4 set_mon_vals no callback set 2023-11-02T15:33:08.709 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:08.710 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.694+0000 7f871affd640 10 monclient: dump: 2023-11-02T15:33:08.710 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.710 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.710 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.711 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.713 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.713 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 10 monclient: shutdown 2023-11-02T15:33:08.713 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:08.713 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 -- 172.21.15.175:0/5028333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 msgr2=0x7f871c13ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.713 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 --2- 172.21.15.175:0/5028333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c13ec30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8714009dc0 tx=0x7f871402bd70 comp rx=0 tx=0).stop 2023-11-02T15:33:08.714 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 -- 172.21.15.175:0/5028333 shutdown_connections 2023-11-02T15:33:08.714 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 --2- 172.21.15.175:0/5028333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c13ec30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.714 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 -- 172.21.15.175:0/5028333 >> 172.21.15.175:0/5028333 conn(0x7f871c065d50 msgr2=0x7f871c066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:08.714 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 -- 172.21.15.175:0/5028333 shutdown_connections 2023-11-02T15:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.695+0000 7f87226db640 1 -- 172.21.15.175:0/5028333 wait complete. 2023-11-02T15:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 10 monclient: build_initial_monmap 2023-11-02T15:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 10 monclient: monmap: 2023-11-02T15:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.716 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 librados: starting msgr at 2023-11-02T15:33:08.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 librados: starting objecter 2023-11-02T15:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 Processor -- start 2023-11-02T15:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 -- start start 2023-11-02T15:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 librados: setting wanted keys 2023-11-02T15:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 1 librados: calling monclient init 2023-11-02T15:33:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.696+0000 7f87226db640 10 monclient: init 2023-11-02T15:33:08.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:08.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:33:08.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c0b0ed0 0x7f871c138a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): picked mon.noname-a con 0x7f871c0b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:08.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c0b4570 0x7f871c138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): picked mon.noname-c con 0x7f871c0b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): picked mon.noname-b con 0x7f871c144d40 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c142250 con 0x7f871c0b0ed0 2023-11-02T15:33:08.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1d63f0 con 0x7f871c144d40 2023-11-02T15:33:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c140370 con 0x7f871c0b4570 2023-11-02T15:33:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f87226db640 10 monclient(hunting): authenticate will time out at 1312.915512s 2023-11-02T15:33:08.722 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.697+0000 7f8720c78640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.722 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c0b0ed0 0x7f871c138a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.722 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49156/0 (socket says 172.21.15.175:49156) 2023-11-02T15:33:08.722 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871b7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c0b4570 0x7f871c138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:08.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 1 -- 172.21.15.175:0/3051496091 learned_addr learned my addr 172.21.15.175:0/3051496091 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:08.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): get_auth_request con 0x7f871c144d40 auth_method 0 2023-11-02T15:33:08.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): get_auth_request con 0x7f871c0b0ed0 auth_method 0 2023-11-02T15:33:08.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871b7fe640 10 monclient(hunting): get_auth_request con 0x7f871c0b4570 auth_method 0 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f8720c78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.698+0000 7f871bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.699+0000 7f871b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:08.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.699+0000 7f871b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:08.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.699+0000 7f871b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:08.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 10 monclient(hunting): handle_auth_done global_id 4126 payload 824 2023-11-02T15:33:08.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 -- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c0b4570 msgr2=0x7f871c138f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c0b4570 0x7f871c138f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 -- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c0b0ed0 msgr2=0x7f871c138a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c0b0ed0 0x7f871c138a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 10 monclient: _finish_hunting 0 2023-11-02T15:33:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 monclient: found mon.noname-b 2023-11-02T15:33:08.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.701+0000 7f8720c78640 1 -- 172.21.15.175:0/3051496091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714009a70 con 0x7f871c144d40 2023-11-02T15:33:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.703+0000 7f8720c78640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8714037040 tx=0x7f8714004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.703+0000 7f8718ff9640 10 client.?.objecter ms_handle_connect 0x7f871c144d40 2023-11-02T15:33:08.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.703+0000 7f8718ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871403e610 con 0x7f871c144d40 2023-11-02T15:33:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: dump: 2023-11-02T15:33:08.732 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.734 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: _finish_auth 0 2023-11-02T15:33:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:33:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871403e770 con 0x7f871c144d40 2023-11-02T15:33:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 5 monclient: authenticate success, global_id 4126 2023-11-02T15:33:08.735 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 10 monclient: _renew_subs 2023-11-02T15:33:08.736 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.736 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c0aaf30 con 0x7f871c144d40 2023-11-02T15:33:08.736 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:08.736 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871403eb00 con 0x7f871c144d40 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f8718ff9640 10 monclient: dump: 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:08.737 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:08.738 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f871a7fc640 4 set_mon_vals no callback set 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 10 client.4126.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:08.739 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 10 monclient: _renew_subs 2023-11-02T15:33:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c1d6fa0 con 0x7f871c144d40 2023-11-02T15:33:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.707+0000 7f87226db640 1 librados: init done 2023-11-02T15:33:08.740 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.708+0000 7f87226db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.708+0000 7f87226db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.708+0000 7f87226db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.741 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.708+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c067920 con 0x7f871c144d40 2023-11-02T15:33:08.771 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.769+0000 7f97c6b51040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-02T15:33:08.788 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.786+0000 7f30d584b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-02T15:33:08.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f871403b050 con 0x7f871c144d40 2023-11-02T15:33:08.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f87140523e0 con 0x7f871c144d40 2023-11-02T15:33:08.807 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 10 client.4126.objecter ms_dispatch 0x7f871c000b90 osd_map(5..5 src has 1..5) v4 2023-11-02T15:33:08.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 3 client.4126.objecter handle_osd_map got epochs [5,5] > 0 2023-11-02T15:33:08.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 3 client.4126.objecter handle_osd_map decoding full epoch 5 2023-11-02T15:33:08.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 20 client.4126.objecter dump_active .. 0 homeless 2023-11-02T15:33:08.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.805+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f871404e060 con 0x7f871c144d40 2023-11-02T15:33:08.809 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.806+0000 7f8718ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:08.809 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.806+0000 7f8718ff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:08.866 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:08.865+0000 7f97c6b51040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-02T15:33:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.876+0000 7f87226db640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "751b6bb7-19d9-47ab-a641-5559f3e6d131", "id": 3}] 2023-11-02T15:33:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.876+0000 7f87226db640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "751b6bb7-19d9-47ab-a641-5559f3e6d131", "id": 3}] 2023-11-02T15:33:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.876+0000 7f87226db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:08.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.876+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "751b6bb7-19d9-47ab-a641-5559f3e6d131", "id": 3} v 0) v1 -- 0x7f871c1d6530 con 0x7f871c144d40 2023-11-02T15:33:08.882 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:08.880+0000 7f30d584b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-02T15:33:08.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.939+0000 7f8718ff9640 1 -- 172.21.15.175:0/3051496091 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "751b6bb7-19d9-47ab-a641-5559f3e6d131", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8714044070 con 0x7f871c144d40 2023-11-02T15:33:08.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.939+0000 7f8718ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "751b6bb7-19d9-47ab-a641-5559f3e6d131", "id": 3}] 2023-11-02T15:33:08.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.939+0000 7f8718ff9640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:08.941 INFO:teuthology.orchestra.run.smithi175.stdout:3 2023-11-02T15:33:08.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 10 librados: watch_flush enter 2023-11-02T15:33:08.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 10 librados: watch_flush exit 2023-11-02T15:33:08.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 20 client.4126.objecter shutdown clearing up homeless session... 2023-11-02T15:33:08.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 10 client.4126.objecter successfully canceled tick 2023-11-02T15:33:08.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 10 monclient: shutdown 2023-11-02T15:33:08.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:08.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 msgr2=0x7f871c1d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:08.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8714037040 tx=0x7f8714004a40 comp rx=0 tx=0).stop 2023-11-02T15:33:08.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 shutdown_connections 2023-11-02T15:33:08.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c144d40 0x7f871c1d6880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c0b4570 0x7f871c138f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 --2- 172.21.15.175:0/3051496091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c0b0ed0 0x7f871c138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:08.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 >> 172.21.15.175:0/3051496091 conn(0x7f871c065d50 msgr2=0x7f871c13dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:08.946 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 shutdown_connections 2023-11-02T15:33:08.946 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 -- 172.21.15.175:0/3051496091 wait complete. 2023-11-02T15:33:08.946 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:08.940+0000 7f87226db640 1 librados: shutdown 2023-11-02T15:33:08.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new a08d368b-40c5-4e74-b125-dcf45d0e9fc4 4 2023-11-02T15:33:09.013 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:09.012+0000 7f97c6b51040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-02T15:33:09.043 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:09.041+0000 7f30d584b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-02T15:33:09.057 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:09.056+0000 7f97c6b51040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-02T15:33:09.084 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:09.083+0000 7f30d584b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-02T15:33:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 10 monclient: get_monmap_and_config 2023-11-02T15:33:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 10 monclient: monmap: 2023-11-02T15:33:09.092 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.092 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.092 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:09.090567+0000 2023-11-02T15:33:09.092 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:09.090567+0000 2023-11-02T15:33:09.092 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.093 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.093 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:09.093 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:09.093 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:09.094 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:09.094 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:09.094 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 1 Processor -- start 2023-11-02T15:33:09.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 1 -- start start 2023-11-02T15:33:09.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.090+0000 7f0d68373640 10 monclient: init 2023-11-02T15:33:09.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2023-11-02T15:33:09.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0d600b4570 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:09.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d600b0ed0 0x7f0d60144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-c con 0x7f0d600b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:09.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d6013e520 0x7f0d6013e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-b con 0x7f0d6013e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:09.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d600a9780 con 0x7f0d600b4570 2023-11-02T15:33:09.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d600b4950 con 0x7f0d600b0ed0 2023-11-02T15:33:09.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600aa1d0 con 0x7f0d6013e520 2023-11-02T15:33:09.100 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.100 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.091+0000 7f0d68373640 10 monclient(hunting): authenticate will time out at 1313.309773s 2023-11-02T15:33:09.100 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d600b0ed0 0x7f0d60144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.100 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d66910640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d6013e520 0x7f0d6013e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.100 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d600b0ed0 0x7f0d60144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49168/0 (socket says 172.21.15.175:49168) 2023-11-02T15:33:09.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 1 -- 172.21.15.175:0/1404316295 learned_addr learned my addr 172.21.15.175:0/1404316295 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 10 monclient(hunting): get_auth_request con 0x7f0d600b0ed0 auth_method 0 2023-11-02T15:33:09.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d6590e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d66910640 10 monclient(hunting): get_auth_request con 0x7f0d6013e520 auth_method 0 2023-11-02T15:33:09.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d66910640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d66910640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.092+0000 7f0d66910640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.103 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d6590e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.103 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d6590e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.103 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d6590e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.103 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.093+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 10 monclient(hunting): handle_auth_done global_id 4129 payload 274 2023-11-02T15:33:09.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 1 -- 172.21.15.175:0/1404316295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d6013e520 msgr2=0x7f0d6013e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:09.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 1 --2- 172.21.15.175:0/1404316295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d6013e520 0x7f0d6013e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 1 -- 172.21.15.175:0/1404316295 >> v1:172.21.15.175:6789/0 conn(0x7f0d600b4570 legacy=0x7f0d600a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 1 monclient: found mon.noname-c 2023-11-02T15:33:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.094+0000 7f0d6590e640 1 -- 172.21.15.175:0/1404316295 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600aa4b0 con 0x7f0d600b0ed0 2023-11-02T15:33:09.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.096+0000 7f0d6590e640 1 --2- 172.21.15.175:0/1404316295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d60144d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009880 tx=0x7f0d5002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=775634ff1c0b915d server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 1 -- 172.21.15.175:0/1404316295 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5002f050 con 0x7f0d600b0ed0 2023-11-02T15:33:09.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:09.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: dump: 2023-11-02T15:33:09.112 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.112 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.112 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.112 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.112 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.113 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.113 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.113 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.113 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.114 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 1 -- 172.21.15.175:0/1404316295 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50004230 con 0x7f0d600b0ed0 2023-11-02T15:33:09.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d68373640 5 monclient: authenticate success, global_id 4129 2023-11-02T15:33:09.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d68373640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:09.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d68373640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:09.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 1 -- 172.21.15.175:0/1404316295 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50003610 con 0x7f0d600b0ed0 2023-11-02T15:33:09.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d68373640 4 set_mon_vals no callback set 2023-11-02T15:33:09.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:09.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.100+0000 7f0d6510d640 10 monclient: dump: 2023-11-02T15:33:09.117 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.117 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.117 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.117 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.118 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.118 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.118 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.118 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.119 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.119 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 10 monclient: shutdown 2023-11-02T15:33:09.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:09.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 -- 172.21.15.175:0/1404316295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 msgr2=0x7f0d60144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.120 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 --2- 172.21.15.175:0/1404316295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d60144d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009880 tx=0x7f0d5002b7a0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.120 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 -- 172.21.15.175:0/1404316295 shutdown_connections 2023-11-02T15:33:09.120 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 --2- 172.21.15.175:0/1404316295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d6013e520 0x7f0d6013e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.120 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 --2- 172.21.15.175:0/1404316295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d60144d40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.120 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 -- 172.21.15.175:0/1404316295 >> 172.21.15.175:0/1404316295 conn(0x7f0d60065d50 msgr2=0x7f0d60066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:09.121 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 -- 172.21.15.175:0/1404316295 shutdown_connections 2023-11-02T15:33:09.121 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.101+0000 7f0d68373640 1 -- 172.21.15.175:0/1404316295 wait complete. 2023-11-02T15:33:09.121 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.122 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 10 monclient: monmap: 2023-11-02T15:33:09.122 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.122 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:09.122 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:09.122 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:09.123 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.123 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.123 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:09.123 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:09.123 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:09.124 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 librados: starting msgr at 2023-11-02T15:33:09.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 librados: starting objecter 2023-11-02T15:33:09.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 Processor -- start 2023-11-02T15:33:09.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 -- start start 2023-11-02T15:33:09.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 librados: setting wanted keys 2023-11-02T15:33:09.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 1 librados: calling monclient init 2023-11-02T15:33:09.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.102+0000 7f0d68373640 10 monclient: init 2023-11-02T15:33:09.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:33:09.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-b con 0x7f0d600b0ed0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:09.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d600b4570 0x7f0d600ab450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-c con 0x7f0d600b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:09.127 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6013e520 0x7f0d600b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.127 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): picked mon.noname-a con 0x7f0d6013e520 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:09.127 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.127 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600ac740 con 0x7f0d6013e520 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601d6960 con 0x7f0d600b0ed0 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600ab990 con 0x7f0d600b4570 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.128 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d68373640 10 monclient(hunting): authenticate will time out at 1313.321257s 2023-11-02T15:33:09.129 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.129 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d66910640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6013e520 0x7f0d600b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.129 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49178/0 (socket says 172.21.15.175:49178) 2023-11-02T15:33:09.129 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 1 -- 172.21.15.175:0/3702994413 learned_addr learned my addr 172.21.15.175:0/3702994413 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.130 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 10 monclient(hunting): get_auth_request con 0x7f0d600b0ed0 auth_method 0 2023-11-02T15:33:09.130 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.130 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.130 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.103+0000 7f0d6610f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.130 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): get_auth_request con 0x7f0d6013e520 auth_method 0 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d6610f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d6610f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.131 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d6610f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.104+0000 7f0d66910640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6590e640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d600b4570 0x7f0d600ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6590e640 10 monclient(hunting): get_auth_request con 0x7f0d600b4570 auth_method 0 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6590e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6590e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6590e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6610f640 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2023-11-02T15:33:09.133 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6610f640 1 -- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d600b4570 msgr2=0x7f0d600ab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6610f640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d600b4570 0x7f0d600ab450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6610f640 1 -- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6013e520 msgr2=0x7f0d600b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.106+0000 7f0d6610f640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6013e520 0x7f0d600b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.107+0000 7f0d6610f640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.107+0000 7f0d6610f640 1 monclient: found mon.noname-b 2023-11-02T15:33:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.107+0000 7f0d6610f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.107+0000 7f0d6610f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.107+0000 7f0d6610f640 1 -- 172.21.15.175:0/3702994413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50009460 con 0x7f0d600b0ed0 2023-11-02T15:33:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.109+0000 7f0d6610f640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c410 tx=0x7f0d5c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.109+0000 7f0d56ffd640 10 client.?.objecter ms_handle_connect 0x7f0d600b0ed0 2023-11-02T15:33:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.109+0000 7f0d56ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0035f0 con 0x7f0d600b0ed0 2023-11-02T15:33:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: dump: 2023-11-02T15:33:09.139 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.139 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.139 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.139 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.139 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.140 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c003750 con 0x7f0d600b0ed0 2023-11-02T15:33:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 5 monclient: authenticate success, global_id 4132 2023-11-02T15:33:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 10 monclient: _renew_subs 2023-11-02T15:33:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d60142330 con 0x7f0d600b0ed0 2023-11-02T15:33:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c003b40 con 0x7f0d600b0ed0 2023-11-02T15:33:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d56ffd640 10 monclient: dump: 2023-11-02T15:33:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.144 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.144 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.144 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.144 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.144 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.145 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.145 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.145 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d6490c640 4 set_mon_vals no callback set 2023-11-02T15:33:09.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 10 client.4132.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:09.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 10 monclient: _renew_subs 2023-11-02T15:33:09.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d60140690 con 0x7f0d600b0ed0 2023-11-02T15:33:09.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.112+0000 7f0d68373640 1 librados: init done 2023-11-02T15:33:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.113+0000 7f0d68373640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.113+0000 7f0d68373640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.113+0000 7f0d68373640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.113+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60067920 con 0x7f0d600b0ed0 2023-11-02T15:33:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0d5c01c440 con 0x7f0d600b0ed0 2023-11-02T15:33:09.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0d5c01b8c0 con 0x7f0d600b0ed0 2023-11-02T15:33:09.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 10 client.4132.objecter ms_dispatch 0x7f0d60000b90 osd_map(6..6 src has 1..6) v4 2023-11-02T15:33:09.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 3 client.4132.objecter handle_osd_map got epochs [6,6] > 0 2023-11-02T15:33:09.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 3 client.4132.objecter handle_osd_map decoding full epoch 6 2023-11-02T15:33:09.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.115+0000 7f0d56ffd640 20 client.4132.objecter dump_active .. 0 homeless 2023-11-02T15:33:09.200 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:09.199+0000 7f97c6b51040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-02T15:33:09.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.209+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d600b0ed0 2023-11-02T15:33:09.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.209+0000 7f0d56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.209+0000 7f0d56ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:09.227 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:09.226+0000 7f30d584b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-02T15:33:09.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.281+0000 7f0d68373640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a08d368b-40c5-4e74-b125-dcf45d0e9fc4", "id": 4}] 2023-11-02T15:33:09.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.281+0000 7f0d68373640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a08d368b-40c5-4e74-b125-dcf45d0e9fc4", "id": 4}] 2023-11-02T15:33:09.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.281+0000 7f0d68373640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.281+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a08d368b-40c5-4e74-b125-dcf45d0e9fc4", "id": 4} v 0) v1 -- 0x7f0d600a8bb0 con 0x7f0d600b0ed0 2023-11-02T15:33:09.351 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.350+0000 7f0d56ffd640 1 -- 172.21.15.175:0/3702994413 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a08d368b-40c5-4e74-b125-dcf45d0e9fc4", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0d5c01b630 con 0x7f0d600b0ed0 2023-11-02T15:33:09.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.350+0000 7f0d56ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a08d368b-40c5-4e74-b125-dcf45d0e9fc4", "id": 4}] 2023-11-02T15:33:09.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.350+0000 7f0d56ffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:09.352 INFO:teuthology.orchestra.run.smithi175.stdout:4 2023-11-02T15:33:09.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 10 librados: watch_flush enter 2023-11-02T15:33:09.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 10 librados: watch_flush exit 2023-11-02T15:33:09.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 20 client.4132.objecter shutdown clearing up homeless session... 2023-11-02T15:33:09.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 10 client.4132.objecter successfully canceled tick 2023-11-02T15:33:09.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 10 monclient: shutdown 2023-11-02T15:33:09.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:09.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 msgr2=0x7f0d600ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c410 tx=0x7f0d5c00c8d0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 shutdown_connections 2023-11-02T15:33:09.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6013e520 0x7f0d600b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d600b4570 0x7f0d600ab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 --2- 172.21.15.175:0/3702994413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d600b0ed0 0x7f0d600ac200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 >> 172.21.15.175:0/3702994413 conn(0x7f0d60065d50 msgr2=0x7f0d600b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:09.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 shutdown_connections 2023-11-02T15:33:09.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 -- 172.21.15.175:0/3702994413 wait complete. 2023-11-02T15:33:09.357 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.351+0000 7f0d68373640 1 librados: shutdown 2023-11-02T15:33:09.362 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 27014a04-b88c-4215-ba2f-9d2865093de5 5 2023-11-02T15:33:09.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.503+0000 7fad66e3a640 10 monclient: get_monmap_and_config 2023-11-02T15:33:09.505 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.503+0000 7fad66e3a640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.505 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.503+0000 7fad66e3a640 10 monclient: monmap: 2023-11-02T15:33:09.505 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.505 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.506 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:09.504092+0000 2023-11-02T15:33:09.506 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:09.504092+0000 2023-11-02T15:33:09.506 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.506 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:09.507 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.509 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.504+0000 7fad66e3a640 1 Processor -- start 2023-11-02T15:33:09.509 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.504+0000 7fad66e3a640 1 -- start start 2023-11-02T15:33:09.509 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.504+0000 7fad66e3a640 10 monclient: init 2023-11-02T15:33:09.510 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.510 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2023-11-02T15:33:09.510 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad600b4570 0x7fad60144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.510 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-c con 0x7fad600b4570 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:09.510 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fad600b0ed0 0x7fad600a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-a con 0x7fad600b0ed0 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:09.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fad600b1cb0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:09.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad600aa1d0 con 0x7fad600b1cb0 2023-11-02T15:33:09.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad600b4950 con 0x7fad600b0ed0 2023-11-02T15:33:09.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad600aa4b0 con 0x7fad600b4570 2023-11-02T15:33:09.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.505+0000 7fad66e3a640 10 monclient(hunting): authenticate will time out at 1313.723419s 2023-11-02T15:33:09.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.509+0000 7fad64bd6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad600b4570 0x7fad60144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad5ffff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fad600b0ed0 0x7fad600a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad600b4570 0x7fad60144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49188/0 (socket says 172.21.15.175:49188) 2023-11-02T15:33:09.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 1 -- 172.21.15.175:0/3532620848 learned_addr learned my addr 172.21.15.175:0/3532620848 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 10 monclient(hunting): get_auth_request con 0x7fad600b4570 auth_method 0 2023-11-02T15:33:09.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad64bd6640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad5ffff640 10 monclient(hunting): get_auth_request con 0x7fad600b0ed0 auth_method 0 2023-11-02T15:33:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad5ffff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.510+0000 7fad5ffff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.511+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.512+0000 7fad64bd6640 10 monclient(hunting): handle_auth_done global_id 4135 payload 274 2023-11-02T15:33:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 1 -- 172.21.15.175:0/3532620848 >> v2:172.21.15.174:3300/0 conn(0x7fad600b0ed0 msgr2=0x7fad600a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 1 --2- 172.21.15.175:0/3532620848 >> v2:172.21.15.174:3300/0 conn(0x7fad600b0ed0 0x7fad600a9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 1 -- 172.21.15.175:0/3532620848 >> v1:172.21.15.174:6789/0 conn(0x7fad600b1cb0 legacy=0x7fad600b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 1 monclient: found mon.noname-c 2023-11-02T15:33:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.513+0000 7fad64bd6640 1 -- 172.21.15.175:0/3532620848 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad600a8b50 con 0x7fad600b4570 2023-11-02T15:33:09.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.514+0000 7fad64bd6640 1 --2- 172.21.15.175:0/3532620848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad60144d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad54009870 tx=0x7fad5402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffffb8225b28ba33 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 1 -- 172.21.15.175:0/3532620848 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540317e0 con 0x7fad600b4570 2023-11-02T15:33:09.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:09.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: dump: 2023-11-02T15:33:09.525 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.525 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.525 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.527 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad66e3a640 5 monclient: authenticate success, global_id 4135 2023-11-02T15:33:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 1 -- 172.21.15.175:0/3532620848 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54031940 con 0x7fad600b4570 2023-11-02T15:33:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad66e3a640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:09.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad66e3a640 4 set_mon_vals no callback set 2023-11-02T15:33:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 1 -- 172.21.15.175:0/3532620848 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54031cd0 con 0x7fad600b4570 2023-11-02T15:33:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.518+0000 7fad5f7fe640 10 monclient: dump: 2023-11-02T15:33:09.529 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.530 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.530 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.530 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.530 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.530 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 10 monclient: shutdown 2023-11-02T15:33:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 -- 172.21.15.175:0/3532620848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 msgr2=0x7fad60144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 --2- 172.21.15.175:0/3532620848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad60144d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad54009870 tx=0x7fad5402b880 comp rx=0 tx=0).stop 2023-11-02T15:33:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 -- 172.21.15.175:0/3532620848 shutdown_connections 2023-11-02T15:33:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 --2- 172.21.15.175:0/3532620848 >> v2:172.21.15.174:3300/0 conn(0x7fad600b0ed0 0x7fad600a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 --2- 172.21.15.175:0/3532620848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad60144d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 -- 172.21.15.175:0/3532620848 >> 172.21.15.175:0/3532620848 conn(0x7fad60065d50 msgr2=0x7fad60066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 -- 172.21.15.175:0/3532620848 shutdown_connections 2023-11-02T15:33:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 1 -- 172.21.15.175:0/3532620848 wait complete. 2023-11-02T15:33:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.519+0000 7fad66e3a640 10 monclient: monmap: 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.535 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:09.535 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:09.535 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:09.535 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.535 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 librados: starting msgr at 2023-11-02T15:33:09.536 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 librados: starting objecter 2023-11-02T15:33:09.536 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 Processor -- start 2023-11-02T15:33:09.536 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 -- start start 2023-11-02T15:33:09.536 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 librados: setting wanted keys 2023-11-02T15:33:09.536 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 1 librados: calling monclient init 2023-11-02T15:33:09.537 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.520+0000 7fad66e3a640 10 monclient: init 2023-11-02T15:33:09.537 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.537 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:33:09.537 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad600b0ed0 0x7fad60144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.537 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-a con 0x7fad600b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:09.538 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad600b1cb0 0x7fad601418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.538 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-c con 0x7fad600b1cb0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:09.538 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.538 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): picked mon.noname-b con 0x7fad600b4570 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:09.538 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60141e10 con 0x7fad600b0ed0 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60142510 con 0x7fad600b4570 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad60142080 con 0x7fad600b1cb0 2023-11-02T15:33:09.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad66e3a640 10 monclient(hunting): authenticate will time out at 1313.739245s 2023-11-02T15:33:09.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad5ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad600b1cb0 0x7fad601418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49200/0 (socket says 172.21.15.175:49200) 2023-11-02T15:33:09.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 1 -- 172.21.15.175:0/1148304296 learned_addr learned my addr 172.21.15.175:0/1148304296 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 10 monclient(hunting): get_auth_request con 0x7fad600b4570 auth_method 0 2023-11-02T15:33:09.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad653d7640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.521+0000 7fad64bd6640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad600b0ed0 0x7fad60144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): get_auth_request con 0x7fad600b0ed0 auth_method 0 2023-11-02T15:33:09.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): get_auth_request con 0x7fad600b1cb0 auth_method 0 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad653d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad653d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad653d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.544 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.544 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.544 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad64bd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.544 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.544 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.545 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.522+0000 7fad5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.547 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 10 monclient(hunting): handle_auth_done global_id 4138 payload 824 2023-11-02T15:33:09.547 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 -- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad600b1cb0 msgr2=0x7fad601418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.547 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad600b1cb0 0x7fad601418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.547 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 -- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad600b0ed0 msgr2=0x7fad60144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad600b0ed0 0x7fad60144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 monclient: found mon.noname-b 2023-11-02T15:33:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.525+0000 7fad653d7640 1 -- 172.21.15.175:0/1148304296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540094d0 con 0x7fad600b4570 2023-11-02T15:33:09.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.527+0000 7fad653d7640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c7d0 tx=0x7fad5800cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.527+0000 7fad5d7fa640 10 client.?.objecter ms_handle_connect 0x7fad600b4570 2023-11-02T15:33:09.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.527+0000 7fad5d7fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:09.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58003b70 con 0x7fad600b4570 2023-11-02T15:33:09.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:09.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: dump: 2023-11-02T15:33:09.550 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.550 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58003cf0 con 0x7fad600b4570 2023-11-02T15:33:09.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800aa90 con 0x7fad600b4570 2023-11-02T15:33:09.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 5 monclient: authenticate success, global_id 4138 2023-11-02T15:33:09.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 10 monclient: _renew_subs 2023-11-02T15:33:09.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad60140f20 con 0x7fad600b4570 2023-11-02T15:33:09.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5effd640 4 set_mon_vals no callback set 2023-11-02T15:33:09.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:09.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad5d7fa640 10 monclient: dump: 2023-11-02T15:33:09.555 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.555 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.555 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.555 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.557 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 10 client.4138.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 10 monclient: _renew_subs 2023-11-02T15:33:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad601d6fa0 con 0x7fad600b4570 2023-11-02T15:33:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.530+0000 7fad66e3a640 1 librados: init done 2023-11-02T15:33:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.531+0000 7fad66e3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.531+0000 7fad66e3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.531+0000 7fad66e3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.531+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad60142210 con 0x7fad600b4570 2023-11-02T15:33:09.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fad580050d0 con 0x7fad600b4570 2023-11-02T15:33:09.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fad580259b0 con 0x7fad600b4570 2023-11-02T15:33:09.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 10 client.4138.objecter ms_dispatch 0x7fad60000b90 osd_map(7..7 src has 1..7) v4 2023-11-02T15:33:09.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 3 client.4138.objecter handle_osd_map got epochs [7,7] > 0 2023-11-02T15:33:09.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 3 client.4138.objecter handle_osd_map decoding full epoch 7 2023-11-02T15:33:09.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 20 client.4138.objecter dump_active .. 0 homeless 2023-11-02T15:33:09.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fad58025c90 con 0x7fad600b4570 2023-11-02T15:33:09.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.628+0000 7fad5d7fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:09.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.699+0000 7fad66e3a640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "27014a04-b88c-4215-ba2f-9d2865093de5", "id": 5}] 2023-11-02T15:33:09.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.699+0000 7fad66e3a640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "27014a04-b88c-4215-ba2f-9d2865093de5", "id": 5}] 2023-11-02T15:33:09.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.699+0000 7fad66e3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:09.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.699+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "27014a04-b88c-4215-ba2f-9d2865093de5", "id": 5} v 0) v1 -- 0x7fad600b3ad0 con 0x7fad600b4570 2023-11-02T15:33:09.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.766+0000 7fad5d7fa640 1 -- 172.21.15.175:0/1148304296 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27014a04-b88c-4215-ba2f-9d2865093de5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fad58025e30 con 0x7fad600b4570 2023-11-02T15:33:09.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.766+0000 7fad5d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "27014a04-b88c-4215-ba2f-9d2865093de5", "id": 5}] 2023-11-02T15:33:09.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.766+0000 7fad5d7fa640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:09.768 INFO:teuthology.orchestra.run.smithi175.stdout:5 2023-11-02T15:33:09.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 10 librados: watch_flush enter 2023-11-02T15:33:09.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 10 librados: watch_flush exit 2023-11-02T15:33:09.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 20 client.4138.objecter shutdown clearing up homeless session... 2023-11-02T15:33:09.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 10 client.4138.objecter successfully canceled tick 2023-11-02T15:33:09.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 10 monclient: shutdown 2023-11-02T15:33:09.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:09.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 msgr2=0x7fad601d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad5800c7d0 tx=0x7fad5800cc90 comp rx=0 tx=0).stop 2023-11-02T15:33:09.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 shutdown_connections 2023-11-02T15:33:09.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad600b4570 0x7fad601d6880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad600b1cb0 0x7fad601418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 --2- 172.21.15.175:0/1148304296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad600b0ed0 0x7fad60144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 >> 172.21.15.175:0/1148304296 conn(0x7fad60065d50 msgr2=0x7fad600b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:09.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 shutdown_connections 2023-11-02T15:33:09.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 -- 172.21.15.175:0/1148304296 wait complete. 2023-11-02T15:33:09.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.767+0000 7fad66e3a640 1 librados: shutdown 2023-11-02T15:33:09.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 1edde1fd-2330-42d8-a7bb-015cebd9d38b 6 2023-11-02T15:33:09.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.877+0000 7ff7483ce640 10 monclient: get_monmap_and_config 2023-11-02T15:33:09.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.877+0000 7ff7483ce640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.877+0000 7ff7483ce640 10 monclient: monmap: 2023-11-02T15:33:09.879 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.879 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.880 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:09.877670+0000 2023-11-02T15:33:09.880 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:09.877670+0000 2023-11-02T15:33:09.880 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.880 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.880 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:09.881 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:09.881 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:09.881 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:09.881 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:09.881 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.877+0000 7ff7483ce640 1 Processor -- start 2023-11-02T15:33:09.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 -- start start 2023-11-02T15:33:09.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient: init 2023-11-02T15:33:09.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient: _add_conns ranks=[2,0,3,1,4] 2023-11-02T15:33:09.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b4570 0x7ff740144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-b con 0x7ff7400b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:09.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff7400b0ed0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:09.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff74013e520 0x7ff74013e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.886 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-a con 0x7ff74013e520 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:09.886 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7400a9780 con 0x7ff7400b0ed0 2023-11-02T15:33:09.886 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.886 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7400b4950 con 0x7ff74013e520 2023-11-02T15:33:09.887 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.887 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400aa1d0 con 0x7ff7400b4570 2023-11-02T15:33:09.887 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.887 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.878+0000 7ff7483ce640 10 monclient(hunting): authenticate will time out at 1314.096904s 2023-11-02T15:33:09.887 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff74013e520 0x7ff74013e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74616a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b4570 0x7ff740144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff74013e520 0x7ff74013e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.175:60918/0 (socket says 172.21.15.175:60918) 2023-11-02T15:33:09.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 1 -- 172.21.15.175:0/2472904384 learned_addr learned my addr 172.21.15.175:0/2472904384 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 10 monclient(hunting): get_auth_request con 0x7ff74013e520 auth_method 0 2023-11-02T15:33:09.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74696b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74616a640 10 monclient(hunting): get_auth_request con 0x7ff7400b4570 auth_method 0 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74616a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74616a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.879+0000 7ff74616a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.880+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 10 monclient(hunting): handle_auth_done global_id 4143 payload 274 2023-11-02T15:33:09.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 1 -- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b4570 msgr2=0x7ff740144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 1 --2- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b4570 0x7ff740144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 1 -- 172.21.15.175:0/2472904384 >> v1:172.21.15.174:6789/0 conn(0x7ff7400b0ed0 legacy=0x7ff7400a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 1 monclient: found mon.noname-a 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:09.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.882+0000 7ff74696b640 1 -- 172.21.15.175:0/2472904384 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7400aa4b0 con 0x7ff74013e520 2023-11-02T15:33:09.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.887+0000 7ff74696b640 1 --2- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff74013e990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009e50 tx=0x7ff73c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd5dbc0e3460d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 1 -- 172.21.15.175:0/2472904384 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c02c9e0 con 0x7ff74013e520 2023-11-02T15:33:09.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:09.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: dump: 2023-11-02T15:33:09.897 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.898 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.898 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.898 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.898 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.898 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 1 -- 172.21.15.175:0/2472904384 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c02cb40 con 0x7ff74013e520 2023-11-02T15:33:09.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 1 -- 172.21.15.175:0/2472904384 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0343f0 con 0x7ff74013e520 2023-11-02T15:33:09.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff7483ce640 5 monclient: authenticate success, global_id 4143 2023-11-02T15:33:09.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff7483ce640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:09.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff7483ce640 4 set_mon_vals no callback set 2023-11-02T15:33:09.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:09.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.890+0000 7ff745168640 10 monclient: dump: 2023-11-02T15:33:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.903 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.903 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.903 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.903 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.903 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.904 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 10 monclient: shutdown 2023-11-02T15:33:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 -- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 msgr2=0x7ff74013e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 --2- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff74013e990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009e50 tx=0x7ff73c02be40 comp rx=0 tx=0).stop 2023-11-02T15:33:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 -- 172.21.15.175:0/2472904384 shutdown_connections 2023-11-02T15:33:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 --2- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff74013e990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 --2- 172.21.15.175:0/2472904384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b4570 0x7ff740144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 -- 172.21.15.175:0/2472904384 >> 172.21.15.175:0/2472904384 conn(0x7ff740065d50 msgr2=0x7ff740066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 -- 172.21.15.175:0/2472904384 shutdown_connections 2023-11-02T15:33:09.906 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 1 -- 172.21.15.175:0/2472904384 wait complete. 2023-11-02T15:33:09.906 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 10 monclient: build_initial_monmap 2023-11-02T15:33:09.906 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.891+0000 7ff7483ce640 10 monclient: monmap: 2023-11-02T15:33:09.906 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:09.906 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:09.907 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:09.907 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:09.907 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:09.907 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.907 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:09.908 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:09.908 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:09.908 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 librados: starting msgr at 2023-11-02T15:33:09.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 librados: starting objecter 2023-11-02T15:33:09.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 Processor -- start 2023-11-02T15:33:09.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 -- start start 2023-11-02T15:33:09.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 librados: setting wanted keys 2023-11-02T15:33:09.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 librados: calling monclient init 2023-11-02T15:33:09.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient: init 2023-11-02T15:33:09.910 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:09.910 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:33:09.910 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b0ed0 0x7ff7401d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.910 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-c con 0x7ff7400b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:09.910 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b4570 0x7ff740141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.911 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-b con 0x7ff7400b4570 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:09.911 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:09.911 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): picked mon.noname-a con 0x7ff74013e520 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:09.911 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.911 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff740141df0 con 0x7ff74013e520 2023-11-02T15:33:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401d99e0 con 0x7ff7400b4570 2023-11-02T15:33:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400b66b0 con 0x7ff7400b0ed0 2023-11-02T15:33:09.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:09.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.892+0000 7ff7483ce640 10 monclient(hunting): authenticate will time out at 1314.110960s 2023-11-02T15:33:09.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff745969640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b4570 0x7ff740141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74616a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b0ed0 0x7ff7401d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:09.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.175:60934/0 (socket says 172.21.15.175:60934) 2023-11-02T15:33:09.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 1 -- 172.21.15.175:0/587931530 learned_addr learned my addr 172.21.15.175:0/587931530 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:09.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 10 monclient(hunting): get_auth_request con 0x7ff74013e520 auth_method 0 2023-11-02T15:33:09.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74696b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff745969640 10 monclient(hunting): get_auth_request con 0x7ff7400b4570 auth_method 0 2023-11-02T15:33:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff745969640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff745969640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff745969640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74616a640 10 monclient(hunting): get_auth_request con 0x7ff7400b0ed0 auth_method 0 2023-11-02T15:33:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74616a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74616a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.893+0000 7ff74616a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74696b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff745969640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff745969640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff745969640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.917 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:09.918 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.894+0000 7ff74616a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 10 monclient(hunting): handle_auth_done global_id 4149 payload 824 2023-11-02T15:33:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 -- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b0ed0 msgr2=0x7ff7401d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b0ed0 0x7ff7401d9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.920 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 -- 172.21.15.175:0/587931530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b4570 msgr2=0x7ff740141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b4570 0x7ff740141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 10 monclient: _finish_hunting 0 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 monclient: found mon.noname-a 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:09.921 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.896+0000 7ff74696b640 1 -- 172.21.15.175:0/587931530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009b30 con 0x7ff74013e520 2023-11-02T15:33:09.922 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.899+0000 7ff74696b640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009e50 tx=0x7ff73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:09.922 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.899+0000 7ff736ffd640 10 client.?.objecter ms_handle_connect 0x7ff74013e520 2023-11-02T15:33:09.922 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.899+0000 7ff736ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:09.922 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c040670 con 0x7ff74013e520 2023-11-02T15:33:09.922 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: dump: 2023-11-02T15:33:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.923 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.924 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.925 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.925 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:09.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:09.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:09.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c040810 con 0x7ff74013e520 2023-11-02T15:33:09.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:09.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c040bc0 con 0x7ff74013e520 2023-11-02T15:33:09.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff7483ce640 5 monclient: authenticate success, global_id 4149 2023-11-02T15:33:09.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff7483ce640 10 monclient: _renew_subs 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff7483ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7400b6870 con 0x7ff74013e520 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff744967640 4 set_mon_vals no callback set 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:09.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.902+0000 7ff736ffd640 10 monclient: dump: 2023-11-02T15:33:09.928 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:09.928 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:09.928 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.928 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:09.928 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:09.929 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:09.929 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:09.929 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:09.929 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:09.929 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.903+0000 7ff7483ce640 10 client.4149.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.903+0000 7ff7483ce640 10 monclient: _renew_subs 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.903+0000 7ff7483ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.903+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7401408b0 con 0x7ff74013e520 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.903+0000 7ff7483ce640 1 librados: init done 2023-11-02T15:33:09.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.904+0000 7ff7483ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.904+0000 7ff7483ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:09.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.904+0000 7ff7483ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:09.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:09.904+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7401d9bf0 con 0x7ff74013e520 2023-11-02T15:33:10.014 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:10.013+0000 7f97c6b51040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-02T15:33:10.016 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.015+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff73c052730 con 0x7ff74013e520 2023-11-02T15:33:10.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff73c040d20 con 0x7ff74013e520 2023-11-02T15:33:10.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 10 client.4149.objecter ms_dispatch 0x7ff740000b90 osd_map(8..8 src has 1..8) v4 2023-11-02T15:33:10.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 3 client.4149.objecter handle_osd_map got epochs [8,8] > 0 2023-11-02T15:33:10.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 3 client.4149.objecter handle_osd_map decoding full epoch 8 2023-11-02T15:33:10.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 20 client.4149.objecter dump_active .. 0 homeless 2023-11-02T15:33:10.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff73c044020 con 0x7ff74013e520 2023-11-02T15:33:10.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.020+0000 7ff736ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:10.039 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:10.037+0000 7f30d584b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-02T15:33:10.092 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.091+0000 7ff7483ce640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "1edde1fd-2330-42d8-a7bb-015cebd9d38b", "id": 6}] 2023-11-02T15:33:10.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.091+0000 7ff7483ce640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "1edde1fd-2330-42d8-a7bb-015cebd9d38b", "id": 6}] 2023-11-02T15:33:10.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.091+0000 7ff7483ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:10.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.091+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1edde1fd-2330-42d8-a7bb-015cebd9d38b", "id": 6} v 0) v1 -- 0x7ff7400a8bb0 con 0x7ff74013e520 2023-11-02T15:33:10.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.141+0000 7ff736ffd640 1 -- 172.21.15.175:0/587931530 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1edde1fd-2330-42d8-a7bb-015cebd9d38b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff73c056020 con 0x7ff74013e520 2023-11-02T15:33:10.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.141+0000 7ff736ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "1edde1fd-2330-42d8-a7bb-015cebd9d38b", "id": 6}] 2023-11-02T15:33:10.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.141+0000 7ff736ffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:10.143 INFO:teuthology.orchestra.run.smithi175.stdout:6 2023-11-02T15:33:10.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.141+0000 7ff7483ce640 10 librados: watch_flush enter 2023-11-02T15:33:10.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 10 librados: watch_flush exit 2023-11-02T15:33:10.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 20 client.4149.objecter shutdown clearing up homeless session... 2023-11-02T15:33:10.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 10 client.4149.objecter successfully canceled tick 2023-11-02T15:33:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 10 monclient: shutdown 2023-11-02T15:33:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 msgr2=0x7ff7401418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009e50 tx=0x7ff73c004a40 comp rx=0 tx=0).stop 2023-11-02T15:33:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 shutdown_connections 2023-11-02T15:33:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74013e520 0x7ff7401418b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7400b4570 0x7ff740141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 --2- 172.21.15.175:0/587931530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7400b0ed0 0x7ff7401d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 >> 172.21.15.175:0/587931530 conn(0x7ff740065d50 msgr2=0x7ff740142890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 shutdown_connections 2023-11-02T15:33:10.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 -- 172.21.15.175:0/587931530 wait complete. 2023-11-02T15:33:10.147 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.142+0000 7ff7483ce640 1 librados: shutdown 2023-11-02T15:33:10.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new a0a6eebf-5d56-4899-83ec-b257dd65d940 7 2023-11-02T15:33:10.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 10 monclient: get_monmap_and_config 2023-11-02T15:33:10.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 10 monclient: build_initial_monmap 2023-11-02T15:33:10.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 10 monclient: monmap: 2023-11-02T15:33:10.252 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:10.253 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.253 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:33:10.251484+0000 2023-11-02T15:33:10.253 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:33:10.251484+0000 2023-11-02T15:33:10.253 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:10.254 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.254 INFO:teuthology.orchestra.run.smithi175.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:10.254 INFO:teuthology.orchestra.run.smithi175.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:10.254 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:10.255 INFO:teuthology.orchestra.run.smithi175.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:10.255 INFO:teuthology.orchestra.run.smithi175.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:10.255 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.258 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 1 Processor -- start 2023-11-02T15:33:10.258 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 1 -- start start 2023-11-02T15:33:10.261 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.251+0000 7f1d47742640 10 monclient: init 2023-11-02T15:33:10.262 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:10.262 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient: _add_conns ranks=[1,2,3,4,0] 2023-11-02T15:33:10.262 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d400b4570 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:10.262 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 0x7f1d40144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-b con 0x7f1d400b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1d4013e520 0x7f1d4013e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-a con 0x7f1d4013e520 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400a9780 con 0x7f1d400b4570 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400b4950 con 0x7f1d4013e520 2023-11-02T15:33:10.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400aa1d0 con 0x7f1d400b0ed0 2023-11-02T15:33:10.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:10.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.252+0000 7f1d47742640 10 monclient(hunting): authenticate will time out at 1314.470678s 2023-11-02T15:33:10.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d454de640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1d400b4570 0x7f1d400a9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:46354/0 (socket says 172.21.15.175:46354) 2023-11-02T15:33:10.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d454de640 1 -- 172.21.15.175:0/770191714 learned_addr learned my addr 172.21.15.175:0/770191714 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-11-02T15:33:10.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d44cdd640 1 --2- 172.21.15.175:0/770191714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 0x7f1d40144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d45cdf640 1 --2- 172.21.15.175:0/770191714 >> v2:172.21.15.174:3300/0 conn(0x7f1d4013e520 0x7f1d4013e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d44cdd640 10 monclient(hunting): get_auth_request con 0x7f1d400b0ed0 auth_method 0 2023-11-02T15:33:10.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d44cdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d44cdd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d44cdd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d45cdf640 10 monclient(hunting): get_auth_request con 0x7f1d4013e520 auth_method 0 2023-11-02T15:33:10.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d45cdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d45cdf640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.253+0000 7f1d45cdf640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.254+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 1 -- 172.21.15.175:0/770191714 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517696216 0 0) 0x7f1d400a9780 con 0x7f1d400b4570 2023-11-02T15:33:10.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 10 monclient(hunting): _negotiate 2023-11-02T15:33:10.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 10 monclient(hunting): my global_id is 4144 2023-11-02T15:33:10.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d37fff640 1 -- 172.21.15.175:0/770191714 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d2c003520 con 0x7f1d400b4570 2023-11-02T15:33:10.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_done global_id 4130 payload 274 2023-11-02T15:33:10.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 1 -- 172.21.15.175:0/770191714 >> v2:172.21.15.174:3300/0 conn(0x7f1d4013e520 msgr2=0x7f1d4013e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:10.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 1 --2- 172.21.15.175:0/770191714 >> v2:172.21.15.174:3300/0 conn(0x7f1d4013e520 0x7f1d4013e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 1 -- 172.21.15.175:0/770191714 >> v1:172.21.15.175:6789/0 conn(0x7f1d400b4570 legacy=0x7f1d400a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 10 monclient: _finish_hunting 0 2023-11-02T15:33:10.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 1 monclient: found mon.noname-b 2023-11-02T15:33:10.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:10.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.255+0000 7f1d44cdd640 1 -- 172.21.15.175:0/770191714 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400aa4b0 con 0x7f1d400b0ed0 2023-11-02T15:33:10.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.257+0000 7f1d44cdd640 1 --2- 172.21.15.175:0/770191714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 0x7f1d40144d40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009790 tx=0x7f1d3002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c32a4022890b8a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:10.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 1 -- 172.21.15.175:0/770191714 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300316f0 con 0x7f1d400b0ed0 2023-11-02T15:33:10.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.277 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:10.278 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: dump: 2023-11-02T15:33:10.278 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:10.278 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.278 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.279 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.280 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: _finish_auth 0 2023-11-02T15:33:10.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: _check_auth_tickets 2023-11-02T15:33:10.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:10.280 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 1 -- 172.21.15.175:0/770191714 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30031850 con 0x7f1d400b0ed0 2023-11-02T15:33:10.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:10.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 1 -- 172.21.15.175:0/770191714 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30031be0 con 0x7f1d400b0ed0 2023-11-02T15:33:10.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 5 monclient: authenticate success, global_id 4130 2023-11-02T15:33:10.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:10.281 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 4 set_mon_vals no callback set 2023-11-02T15:33:10.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:10.282 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d37fff640 10 monclient: dump: 2023-11-02T15:33:10.283 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:10.283 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.283 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.283 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.283 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.284 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.284 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.284 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.284 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.284 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 10 monclient: shutdown 2023-11-02T15:33:10.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:10.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 -- 172.21.15.175:0/770191714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 msgr2=0x7f1d40144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 --2- 172.21.15.175:0/770191714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 0x7f1d40144d40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009790 tx=0x7f1d3002b790 comp rx=0 tx=0).stop 2023-11-02T15:33:10.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 -- 172.21.15.175:0/770191714 shutdown_connections 2023-11-02T15:33:10.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 --2- 172.21.15.175:0/770191714 >> v2:172.21.15.174:3300/0 conn(0x7f1d4013e520 0x7f1d4013e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 --2- 172.21.15.175:0/770191714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b0ed0 0x7f1d40144d40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.261+0000 7f1d47742640 1 -- 172.21.15.175:0/770191714 >> 172.21.15.175:0/770191714 conn(0x7f1d40065d50 msgr2=0x7f1d40066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:10.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 1 -- 172.21.15.175:0/770191714 shutdown_connections 2023-11-02T15:33:10.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 1 -- 172.21.15.175:0/770191714 wait complete. 2023-11-02T15:33:10.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 10 monclient: build_initial_monmap 2023-11-02T15:33:10.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 10 monclient: monmap: 2023-11-02T15:33:10.287 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 0 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 0.000000 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:created 0.000000 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.288 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:10.289 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:10.289 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:10.289 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 1 librados: starting msgr at 2023-11-02T15:33:10.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 1 librados: starting objecter 2023-11-02T15:33:10.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.262+0000 7f1d47742640 1 Processor -- start 2023-11-02T15:33:10.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 -- start start 2023-11-02T15:33:10.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 librados: setting wanted keys 2023-11-02T15:33:10.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 librados: calling monclient init 2023-11-02T15:33:10.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient: init 2023-11-02T15:33:10.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:10.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:33:10.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d400b0ed0 0x7f1d400ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-a con 0x7f1d400b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:10.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b4570 0x7f1d400abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-c con 0x7f1d400b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:10.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): picked mon.noname-b con 0x7f1d4013e520 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:10.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401d6590 con 0x7f1d400b0ed0 2023-11-02T15:33:10.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400ad6a0 con 0x7f1d4013e520 2023-11-02T15:33:10.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40140790 con 0x7f1d400b4570 2023-11-02T15:33:10.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:10.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.263+0000 7f1d47742640 10 monclient(hunting): authenticate will time out at 1314.481705s 2023-11-02T15:33:10.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b4570 0x7f1d400abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.294 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d454de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d400b0ed0 0x7f1d400ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:49218/0 (socket says 172.21.15.175:49218) 2023-11-02T15:33:10.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 1 -- 172.21.15.175:0/1450159096 learned_addr learned my addr 172.21.15.175:0/1450159096 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-11-02T15:33:10.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): get_auth_request con 0x7f1d4013e520 auth_method 0 2023-11-02T15:33:10.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): get_auth_request con 0x7f1d400b4570 auth_method 0 2023-11-02T15:33:10.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d454de640 10 monclient(hunting): get_auth_request con 0x7f1d400b0ed0 auth_method 0 2023-11-02T15:33:10.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d454de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d454de640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d454de640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.297 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.264+0000 7f1d44cdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.265+0000 7f1d454de640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.265+0000 7f1d454de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.265+0000 7f1d454de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 -- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b4570 msgr2=0x7f1d400abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b4570 0x7f1d400abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 -- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d400b0ed0 msgr2=0x7f1d400ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d400b0ed0 0x7f1d400ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 10 monclient: _finish_hunting 0 2023-11-02T15:33:10.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 monclient: found mon.noname-b 2023-11-02T15:33:10.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:10.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:10.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.267+0000 7f1d45cdf640 1 -- 172.21.15.175:0/1450159096 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300093a0 con 0x7f1d4013e520 2023-11-02T15:33:10.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.269+0000 7f1d45cdf640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800efc0 tx=0x7f1d3800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:10.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.269+0000 7f1d35ffb640 10 client.?.objecter ms_handle_connect 0x7f1d4013e520 2023-11-02T15:33:10.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.269+0000 7f1d35ffb640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38009ba0 con 0x7f1d4013e520 2023-11-02T15:33:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: dump: 2023-11-02T15:33:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:10.303 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.303 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.303 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.303 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.303 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.304 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.304 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.304 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.304 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: _finish_auth 0 2023-11-02T15:33:10.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:33:10.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:10.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3800f040 con 0x7f1d4013e520 2023-11-02T15:33:10.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:10.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380146c0 con 0x7f1d4013e520 2023-11-02T15:33:10.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:10.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d35ffb640 10 monclient: dump: 2023-11-02T15:33:10.306 INFO:teuthology.orchestra.run.smithi175.stderr:epoch 1 2023-11-02T15:33:10.306 INFO:teuthology.orchestra.run.smithi175.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.307 INFO:teuthology.orchestra.run.smithi175.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.307 INFO:teuthology.orchestra.run.smithi175.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.307 INFO:teuthology.orchestra.run.smithi175.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.307 INFO:teuthology.orchestra.run.smithi175.stderr:election_strategy: 1 2023-11-02T15:33:10.308 INFO:teuthology.orchestra.run.smithi175.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.308 INFO:teuthology.orchestra.run.smithi175.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.308 INFO:teuthology.orchestra.run.smithi175.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.308 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-11-02T15:33:10.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 5 monclient: authenticate success, global_id 4147 2023-11-02T15:33:10.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 10 monclient: _renew_subs 2023-11-02T15:33:10.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:10.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d377fe640 4 set_mon_vals no callback set 2023-11-02T15:33:10.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400a8c00 con 0x7f1d4013e520 2023-11-02T15:33:10.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 10 client.4147.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:10.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 10 monclient: _renew_subs 2023-11-02T15:33:10.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:10.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401d6f20 con 0x7f1d4013e520 2023-11-02T15:33:10.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.273+0000 7f1d47742640 1 librados: init done 2023-11-02T15:33:10.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.274+0000 7f1d47742640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.274+0000 7f1d47742640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.274+0000 7f1d47742640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:10.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.274+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d401d6700 con 0x7f1d4013e520 2023-11-02T15:33:10.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.276+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d38014b20 con 0x7f1d4013e520 2023-11-02T15:33:10.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.276+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1d38026cb0 con 0x7f1d4013e520 2023-11-02T15:33:10.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.276+0000 7f1d35ffb640 10 client.4147.objecter ms_dispatch 0x7f1d40000b90 osd_map(9..9 src has 1..9) v4 2023-11-02T15:33:10.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.276+0000 7f1d35ffb640 3 client.4147.objecter handle_osd_map got epochs [9,9] > 0 2023-11-02T15:33:10.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.276+0000 7f1d35ffb640 3 client.4147.objecter handle_osd_map decoding full epoch 9 2023-11-02T15:33:10.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.277+0000 7f1d35ffb640 20 client.4147.objecter dump_active .. 0 homeless 2023-11-02T15:33:10.314 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:10.293+0000 7f97c6b51040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-02T15:33:10.318 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:10.316+0000 7f30d584b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-02T15:33:10.345 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:10.344+0000 7f97c6b51040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-02T15:33:10.367 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:10.366+0000 7f30d584b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-02T15:33:10.372 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.372+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1d401d6700 con 0x7f1d4013e520 2023-11-02T15:33:10.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.372+0000 7f1d35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.372+0000 7f1d35ffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:10.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.441+0000 7f1d47742640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a0a6eebf-5d56-4899-83ec-b257dd65d940", "id": 7}] 2023-11-02T15:33:10.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.441+0000 7f1d47742640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a0a6eebf-5d56-4899-83ec-b257dd65d940", "id": 7}] 2023-11-02T15:33:10.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.441+0000 7f1d47742640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:10.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.441+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a0a6eebf-5d56-4899-83ec-b257dd65d940", "id": 7} v 0) v1 -- 0x7f1d400ad7e0 con 0x7f1d4013e520 2023-11-02T15:33:10.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-11-02T15:33:10.446+0000 7f97c6b51040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-02T15:33:10.466 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:10.465+0000 7f30d584b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-02T15:33:10.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.612+0000 7f1d35ffb640 1 -- 172.21.15.175:0/1450159096 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a0a6eebf-5d56-4899-83ec-b257dd65d940", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1d400ad7e0 con 0x7f1d4013e520 2023-11-02T15:33:10.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.612+0000 7f1d35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a0a6eebf-5d56-4899-83ec-b257dd65d940", "id": 7}] 2023-11-02T15:33:10.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.612+0000 7f1d35ffb640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:10.614 INFO:teuthology.orchestra.run.smithi175.stdout:7 2023-11-02T15:33:10.615 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 10 librados: watch_flush enter 2023-11-02T15:33:10.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 10 librados: watch_flush exit 2023-11-02T15:33:10.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 20 client.4147.objecter shutdown clearing up homeless session... 2023-11-02T15:33:10.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 10 client.4147.objecter successfully canceled tick 2023-11-02T15:33:10.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 10 monclient: shutdown 2023-11-02T15:33:10.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:10.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 msgr2=0x7f1d401d69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800efc0 tx=0x7f1d3800c490 comp rx=0 tx=0).stop 2023-11-02T15:33:10.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 shutdown_connections 2023-11-02T15:33:10.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d4013e520 0x7f1d401d69e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d400b4570 0x7f1d400abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.618 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 --2- 172.21.15.175:0/1450159096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d400b0ed0 0x7f1d400ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.618 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 >> 172.21.15.175:0/1450159096 conn(0x7f1d40065d50 msgr2=0x7f1d4013c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:10.618 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 shutdown_connections 2023-11-02T15:33:10.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 -- 172.21.15.175:0/1450159096 wait complete. 2023-11-02T15:33:10.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-11-02T15:33:10.613+0000 7f1d47742640 1 librados: shutdown 2023-11-02T15:33:10.624 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.624 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-02T15:33:10.625 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2023-11-02T15:33:10.627 INFO:tasks.ceph.osd.0:Started 2023-11-02T15:33:10.628 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.628 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-02T15:33:10.628 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2023-11-02T15:33:10.630 INFO:tasks.ceph.osd.1:Started 2023-11-02T15:33:10.631 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.631 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-02T15:33:10.632 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2023-11-02T15:33:10.634 INFO:tasks.ceph.osd.2:Started 2023-11-02T15:33:10.634 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.635 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-02T15:33:10.635 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2023-11-02T15:33:10.637 INFO:tasks.ceph.osd.3:Started 2023-11-02T15:33:10.637 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.638 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-02T15:33:10.638 DEBUG:teuthology.orchestra.run.smithi175:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2023-11-02T15:33:10.640 INFO:tasks.ceph.osd.4:Started 2023-11-02T15:33:10.641 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.641 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-02T15:33:10.641 DEBUG:teuthology.orchestra.run.smithi175:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2023-11-02T15:33:10.643 INFO:tasks.ceph.osd.5:Started 2023-11-02T15:33:10.644 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.644 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-02T15:33:10.644 DEBUG:teuthology.orchestra.run.smithi175:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2023-11-02T15:33:10.646 INFO:tasks.ceph.osd.6:Started 2023-11-02T15:33:10.647 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-11-02T15:33:10.647 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-02T15:33:10.647 DEBUG:teuthology.orchestra.run.smithi175:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-11-02T15:33:10.650 INFO:tasks.ceph.osd.7:Started 2023-11-02T15:33:10.650 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:10.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.814+0000 7fc69b25f640 10 monclient: get_monmap_and_config 2023-11-02T15:33:10.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.814+0000 7fc69b25f640 10 monclient: build_initial_monmap 2023-11-02T15:33:10.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.814+0000 7fc69b25f640 10 monclient: monmap: 2023-11-02T15:33:10.817 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:33:10.817 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.817 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:33:10.815333+0000 2023-11-02T15:33:10.818 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:33:10.815333+0000 2023-11-02T15:33:10.818 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:10.818 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:10.818 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:10.818 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:10.819 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:10.819 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:10.819 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:10.819 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:10.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.815+0000 7fc69b25f640 1 Processor -- start 2023-11-02T15:33:10.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.816+0000 7fc69b25f640 1 -- start start 2023-11-02T15:33:10.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.816+0000 7fc69b25f640 10 monclient: init 2023-11-02T15:33:10.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:10.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient: _add_conns ranks=[2,3,1,0,4] 2023-11-02T15:33:10.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-b con 0x7fc6940b4260 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:10.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc694141730 0x7fc694141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-a con 0x7fc694141730 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:10.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc6940b1a00 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:10.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940a9210 con 0x7fc6940b1a00 2023-11-02T15:33:10.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6940aa550 con 0x7fc694141730 2023-11-02T15:33:10.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940b4640 con 0x7fc6940b4260 2023-11-02T15:33:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.817+0000 7fc69b25f640 10 monclient(hunting): authenticate will time out at 1318.159624s 2023-11-02T15:33:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.819+0000 7fc698ffb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:49682/0 (socket says 172.21.15.174:49682) 2023-11-02T15:33:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 1 -- 172.21.15.174:0/3188906154 learned_addr learned my addr 172.21.15.174:0/3188906154 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:10.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): get_auth_request con 0x7fc6940b4260 auth_method 0 2023-11-02T15:33:10.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.820+0000 7fc698ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 10 monclient(hunting): handle_auth_done global_id 4136 payload 274 2023-11-02T15:33:10.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 1 -- 172.21.15.174:0/3188906154 >> v2:172.21.15.174:3300/0 conn(0x7fc694141730 msgr2=0x7fc694141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:10.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 1 --2- 172.21.15.174:0/3188906154 >> v2:172.21.15.174:3300/0 conn(0x7fc694141730 0x7fc694141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 1 -- 172.21.15.174:0/3188906154 >> v1:172.21.15.175:6789/0 conn(0x7fc6940b1a00 legacy=0x7fc6940b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:10.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 10 monclient: _finish_hunting 0 2023-11-02T15:33:10.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 1 monclient: found mon.noname-b 2023-11-02T15:33:10.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:10.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.823+0000 7fc698ffb640 1 -- 172.21.15.174:0/3188906154 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940aa750 con 0x7fc6940b4260 2023-11-02T15:33:10.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.826+0000 7fc698ffb640 1 --2- 172.21.15.174:0/3188906154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc680009870 tx=0x7fc68002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5564df014f1051 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:10.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 1 -- 172.21.15.174:0/3188906154 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6800317c0 con 0x7fc6940b4260 2023-11-02T15:33:10.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:10.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 10 monclient: dump: 2023-11-02T15:33:10.837 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:10.837 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.837 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.837 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.837 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.838 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:10.838 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.838 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.838 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.838 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc6937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc69b25f640 5 monclient: authenticate success, global_id 4136 2023-11-02T15:33:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.829+0000 7fc69b25f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:10.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.832+0000 7fc6937fe640 1 -- 172.21.15.174:0/3188906154 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680031920 con 0x7fc6940b4260 2023-11-02T15:33:10.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.832+0000 7fc6937fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:10.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.832+0000 7fc69b25f640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:10.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.832+0000 7fc69b25f640 4 set_mon_vals no callback set 2023-11-02T15:33:10.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 10 monclient: shutdown 2023-11-02T15:33:10.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:10.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 -- 172.21.15.174:0/3188906154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 msgr2=0x7fc6940a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 --2- 172.21.15.174:0/3188906154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc680009870 tx=0x7fc68002b860 comp rx=0 tx=0).stop 2023-11-02T15:33:10.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 -- 172.21.15.174:0/3188906154 shutdown_connections 2023-11-02T15:33:10.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 --2- 172.21.15.174:0/3188906154 >> v2:172.21.15.174:3300/0 conn(0x7fc694141730 0x7fc694141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 --2- 172.21.15.174:0/3188906154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc6940b4260 0x7fc6940a8870 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 -- 172.21.15.174:0/3188906154 >> 172.21.15.174:0/3188906154 conn(0x7fc694065d50 msgr2=0x7fc694066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:10.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 -- 172.21.15.174:0/3188906154 shutdown_connections 2023-11-02T15:33:10.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.833+0000 7fc69b25f640 1 -- 172.21.15.174:0/3188906154 wait complete. 2023-11-02T15:33:10.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.835+0000 7fc69b25f640 10 monclient: build_initial_monmap 2023-11-02T15:33:10.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.835+0000 7fc69b25f640 10 monclient: monmap: 2023-11-02T15:33:10.848 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:33:10.848 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:10.848 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:10.849 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:10.850 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:10.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.835+0000 7fc69b25f640 1 librados: starting msgr at 2023-11-02T15:33:10.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.835+0000 7fc69b25f640 1 librados: starting objecter 2023-11-02T15:33:10.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.836+0000 7fc69b25f640 1 Processor -- start 2023-11-02T15:33:10.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.836+0000 7fc69b25f640 1 -- start start 2023-11-02T15:33:10.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.836+0000 7fc69b25f640 1 librados: setting wanted keys 2023-11-02T15:33:10.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.836+0000 7fc69b25f640 1 librados: calling monclient init 2023-11-02T15:33:10.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.836+0000 7fc69b25f640 10 monclient: init 2023-11-02T15:33:10.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:10.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:10.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6940b1a00 0x7fc6941cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-b con 0x7fc6940b1a00 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:10.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 0x7fc694137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.837+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-a con 0x7fc6940b4260 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:10.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 0x7fc6941d6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:10.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): picked mon.noname-c con 0x7fc694141730 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:10.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941ccba0 con 0x7fc6940b4260 2023-11-02T15:33:10.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc694138100 con 0x7fc6940b1a00 2023-11-02T15:33:10.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:10.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc694137b70 con 0x7fc694141730 2023-11-02T15:33:10.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:10.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.838+0000 7fc69b25f640 10 monclient(hunting): authenticate will time out at 1318.180189s 2023-11-02T15:33:10.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.839+0000 7fc693fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 0x7fc694137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc6997fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 0x7fc6941d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc693fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 0x7fc694137630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:34696/0 (socket says 172.21.15.174:34696) 2023-11-02T15:33:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc693fff640 1 -- 172.21.15.174:0/2930069072 learned_addr learned my addr 172.21.15.174:0/2930069072 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc693fff640 10 monclient(hunting): get_auth_request con 0x7fc6940b4260 auth_method 0 2023-11-02T15:33:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc693fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.840+0000 7fc693fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc693fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): get_auth_request con 0x7fc694141730 auth_method 0 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:10.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:10.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:10.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:10.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.841+0000 7fc6997fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:10.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 10 monclient(hunting): handle_auth_done global_id 4139 payload 824 2023-11-02T15:33:10.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 -- 172.21.15.174:0/2930069072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6940b1a00 msgr2=0x7fc6941cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6940b1a00 0x7fc6941cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 -- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 msgr2=0x7fc694137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:10.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 0x7fc694137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:10.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 10 monclient: _finish_hunting 0 2023-11-02T15:33:10.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 monclient: found mon.noname-c 2023-11-02T15:33:10.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:10.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.846+0000 7fc6997fc640 1 -- 172.21.15.174:0/2930069072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800094d0 con 0x7fc694141730 2023-11-02T15:33:10.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.849+0000 7fc6997fc640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 0x7fc6941d6980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c780 tx=0x7fc68c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:10.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.849+0000 7fc6917fa640 10 client.?.objecter ms_handle_connect 0x7fc694141730 2023-11-02T15:33:10.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.849+0000 7fc6917fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:10.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003b70 con 0x7fc694141730 2023-11-02T15:33:10.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:10.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: dump: 2023-11-02T15:33:10.866 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:10.866 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.866 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.866 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.867 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.867 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:10.867 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.867 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.868 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.868 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:10.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: _finish_auth 0 2023-11-02T15:33:10.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: _check_auth_tickets 2023-11-02T15:33:10.869 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:10.869 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c003cf0 con 0x7fc694141730 2023-11-02T15:33:10.869 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:10.869 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003ea0 con 0x7fc694141730 2023-11-02T15:33:10.869 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc6917fa640 10 monclient: dump: 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.870 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:10.871 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:10.871 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:10.871 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:10.871 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:10.871 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc692ffd640 4 set_mon_vals no callback set 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc69b25f640 5 monclient: authenticate success, global_id 4139 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc69b25f640 10 monclient: _renew_subs 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc69b25f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc694137d90 con 0x7fc694141730 2023-11-02T15:33:10.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.855+0000 7fc69b25f640 10 client.4139.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:10.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.856+0000 7fc69b25f640 10 monclient: _renew_subs 2023-11-02T15:33:10.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.856+0000 7fc69b25f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.856+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6941d70a0 con 0x7fc694141730 2023-11-02T15:33:10.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.856+0000 7fc69b25f640 1 librados: init done 2023-11-02T15:33:10.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.857+0000 7fc69b25f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.857+0000 7fc69b25f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:10.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.857+0000 7fc69b25f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:10.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.857+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6941382c0 con 0x7fc694141730 2023-11-02T15:33:10.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.876+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 2) v1 ==== 44255+0+0 (secure 0 0 0) 0x7fc68c006b10 con 0x7fc694141730 2023-11-02T15:33:10.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.996+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc68c006dc0 con 0x7fc694141730 2023-11-02T15:33:10.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.996+0000 7fc6917fa640 10 client.4139.objecter ms_dispatch 0x7fc694000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:10.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.996+0000 7fc6917fa640 3 client.4139.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:10.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.996+0000 7fc6917fa640 3 client.4139.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:10.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:10.996+0000 7fc6917fa640 20 client.4139.objecter dump_active .. 0 homeless 2023-11-02T15:33:11.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.057+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc68c0057d0 con 0x7fc694141730 2023-11-02T15:33:11.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.057+0000 7fc6917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:11.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.057+0000 7fc6917fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:11.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.178+0000 7fc69b25f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:11.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.178+0000 7fc69b25f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:11.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.178+0000 7fc69b25f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:11.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.178+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc694067920 con 0x7fc694141730 2023-11-02T15:33:11.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.250+0000 7fc6917fa640 1 -- 172.21.15.174:0/2930069072 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc68c017030 con 0x7fc694141730 2023-11-02T15:33:11.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.250+0000 7fc6917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:11.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.250+0000 7fc6917fa640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:11.253 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:33:11.253 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:11.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 10 librados: watch_flush enter 2023-11-02T15:33:11.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 10 librados: watch_flush exit 2023-11-02T15:33:11.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 20 client.4139.objecter shutdown clearing up homeless session... 2023-11-02T15:33:11.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 10 client.4139.objecter successfully canceled tick 2023-11-02T15:33:11.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 10 monclient: shutdown 2023-11-02T15:33:11.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 msgr2=0x7fc6941d6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 0x7fc6941d6980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c780 tx=0x7fc68c00cc40 comp rx=0 tx=0).stop 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 shutdown_connections 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc694141730 0x7fc6941d6980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6940b4260 0x7fc694137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:11.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 --2- 172.21.15.174:0/2930069072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6940b1a00 0x7fc6941cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:11.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.252+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 >> 172.21.15.174:0/2930069072 conn(0x7fc694065d50 msgr2=0x7fc69413b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:11.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.253+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 shutdown_connections 2023-11-02T15:33:11.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.253+0000 7fc69b25f640 1 -- 172.21.15.174:0/2930069072 wait complete. 2023-11-02T15:33:11.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:11.253+0000 7fc69b25f640 1 librados: shutdown 2023-11-02T15:33:11.282 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-02T15:33:11.282 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-02T15:33:12.089 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:12.244 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.242+0000 7f8cae6f7640 10 monclient: get_monmap_and_config 2023-11-02T15:33:12.244 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.242+0000 7f8cae6f7640 10 monclient: build_initial_monmap 2023-11-02T15:33:12.245 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.242+0000 7f8cae6f7640 10 monclient: monmap: 2023-11-02T15:33:12.245 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:12.245 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:12.246 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:12.242978+0000 2023-11-02T15:33:12.246 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:12.242978+0000 2023-11-02T15:33:12.247 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:12.247 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.247 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:12.248 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:12.248 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:12.248 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:12.249 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:12.249 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.253 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.243+0000 7f8cae6f7640 1 Processor -- start 2023-11-02T15:33:12.253 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.243+0000 7f8cae6f7640 1 -- start start 2023-11-02T15:33:12.254 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.243+0000 7f8cae6f7640 10 monclient: init 2023-11-02T15:33:12.254 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:12.255 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2023-11-02T15:33:12.255 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8ca80b41c0 0x7f8ca80a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.255 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-a con 0x7f8ca80b41c0 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:12.255 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca813e780 0x7f8ca813ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.256 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-b con 0x7f8ca813e780 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:12.256 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ca81416a0 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:12.256 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80aa360 con 0x7f8ca81416a0 2023-11-02T15:33:12.257 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:12.257 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca813f100 con 0x7f8ca80b41c0 2023-11-02T15:33:12.257 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:12.257 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80aa160 con 0x7f8ca813e780 2023-11-02T15:33:12.258 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:12.258 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.244+0000 7f8cae6f7640 10 monclient(hunting): authenticate will time out at 1319.586040s 2023-11-02T15:33:12.262 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8ca80b41c0 0x7f8ca80a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:12.263 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8ca80b41c0 0x7f8ca80a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:34794/0 (socket says 172.21.15.174:34794) 2023-11-02T15:33:12.263 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 1 -- 172.21.15.174:0/2186872038 learned_addr learned my addr 172.21.15.174:0/2186872038 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:12.263 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 10 monclient(hunting): get_auth_request con 0x7f8ca80b41c0 auth_method 0 2023-11-02T15:33:12.264 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:12.264 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:12.264 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.260+0000 7f8ca7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:12.334 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.333+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:12.335 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.333+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:12.335 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.333+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:12.402 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.401+0000 7f8ca77fe640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca813e780 0x7f8ca813ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:12.410 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_done global_id 4170 payload 274 2023-11-02T15:33:12.411 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 1 -- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca813e780 msgr2=0x7f8ca813ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:12.411 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca813e780 0x7f8ca813ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.411 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 1 -- 172.21.15.174:0/2186872038 >> v1:172.21.15.175:6789/0 conn(0x7f8ca81416a0 legacy=0x7f8ca8141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:12.411 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 10 monclient: _finish_hunting 0 2023-11-02T15:33:12.411 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 1 monclient: found mon.noname-a 2023-11-02T15:33:12.412 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:12.412 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:12.412 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.408+0000 7f8ca7fff640 1 -- 172.21.15.174:0/2186872038 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80aa640 con 0x7f8ca80b41c0 2023-11-02T15:33:12.418 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.417+0000 7f8ca7fff640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80a8870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009870 tx=0x7f8c9802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d42f8855a88d8d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:12.499 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:12.498+0000 7f3076cbc640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-02T15:33:12.563 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T15:33:12.561+0000 7f3076cbc640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-02T15:33:12.656 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 1 -- 172.21.15.174:0/2186872038 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98031740 con 0x7f8ca80b41c0 2023-11-02T15:33:12.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:12.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:12.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: dump: 2023-11-02T15:33:12.658 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:12.658 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:12.658 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.659 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.659 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:12.659 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.660 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:12.660 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:12.660 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:12.661 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.661 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:12.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:12.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:12.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 1 -- 172.21.15.174:0/2186872038 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c980318a0 con 0x7f8ca80b41c0 2023-11-02T15:33:12.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8cae6f7640 5 monclient: authenticate success, global_id 4170 2023-11-02T15:33:12.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8cae6f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:12.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:12.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 1 -- 172.21.15.174:0/2186872038 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98031c30 con 0x7f8ca80b41c0 2023-11-02T15:33:12.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8cae6f7640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:12.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8cae6f7640 4 set_mon_vals no callback set 2023-11-02T15:33:12.664 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:12.664 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:12.664 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.653+0000 7f8ca6ffd640 10 monclient: dump: 2023-11-02T15:33:12.664 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:12.664 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:12.665 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.665 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.665 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:12.665 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.666 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:12.666 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:12.666 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:12.667 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.667 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 10 monclient: shutdown 2023-11-02T15:33:12.667 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:12.667 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 -- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca80b41c0 msgr2=0x7f8ca80a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:12.668 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80a8870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009870 tx=0x7f8c9802b7e0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.668 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 -- 172.21.15.174:0/2186872038 shutdown_connections 2023-11-02T15:33:12.668 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca813e780 0x7f8ca813ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.668 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/2186872038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80a8870 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.668 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 -- 172.21.15.174:0/2186872038 >> 172.21.15.174:0/2186872038 conn(0x7f8ca8065e10 msgr2=0x7f8ca8066220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:12.669 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 -- 172.21.15.174:0/2186872038 shutdown_connections 2023-11-02T15:33:12.669 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.654+0000 7f8cae6f7640 1 -- 172.21.15.174:0/2186872038 wait complete. 2023-11-02T15:33:12.672 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.655+0000 7f8cae6f7640 10 monclient: build_initial_monmap 2023-11-02T15:33:12.672 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.655+0000 7f8cae6f7640 10 monclient: monmap: 2023-11-02T15:33:12.673 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:12.673 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:12.673 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:12.673 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:12.674 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:12.674 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.674 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:12.674 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:12.675 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:12.675 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.675 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.655+0000 7f8cae6f7640 1 librados: starting msgr at 2023-11-02T15:33:12.675 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.655+0000 7f8cae6f7640 1 librados: starting objecter 2023-11-02T15:33:12.676 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.656+0000 7f8cae6f7640 1 Processor -- start 2023-11-02T15:33:12.676 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.656+0000 7f8cae6f7640 1 -- start start 2023-11-02T15:33:12.676 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.656+0000 7f8cae6f7640 1 librados: setting wanted keys 2023-11-02T15:33:12.676 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.656+0000 7f8cae6f7640 1 librados: calling monclient init 2023-11-02T15:33:12.677 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.656+0000 7f8cae6f7640 10 monclient: init 2023-11-02T15:33:12.677 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:12.677 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:12.677 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.678 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-b con 0x7f8ca80b41c0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:12.678 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca813e780 0x7f8ca8131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.678 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-a con 0x7f8ca813e780 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:12.678 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca81416a0 0x7f8ca812c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.679 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient(hunting): picked mon.noname-c con 0x7f8ca81416a0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:12.679 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:12.679 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.657+0000 7f8cae6f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80b7c60 con 0x7f8ca813e780 2023-11-02T15:33:12.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:12.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81329f0 con 0x7f8ca80b41c0 2023-11-02T15:33:12.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:12.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8132cd0 con 0x7f8ca81416a0 2023-11-02T15:33:12.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:12.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:12.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cacc94640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca81416a0 0x7f8ca812c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:12.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:39054/0 (socket says 172.21.15.174:39054) 2023-11-02T15:33:12.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 1 -- 172.21.15.174:0/775431227 learned_addr learned my addr 172.21.15.174:0/775431227 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:12.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cae6f7640 10 monclient(hunting): authenticate will time out at 1320.000445s 2023-11-02T15:33:12.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 10 monclient(hunting): get_auth_request con 0x7f8ca80b41c0 auth_method 0 2023-11-02T15:33:12.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:12.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:12.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8ca7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:12.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cacc94640 10 monclient(hunting): get_auth_request con 0x7f8ca81416a0 auth_method 0 2023-11-02T15:33:12.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cacc94640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:12.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cacc94640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:12.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.658+0000 7f8cacc94640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:12.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:12.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:12.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:12.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8cacc94640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:12.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8cacc94640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:12.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.659+0000 7f8cacc94640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:12.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.663+0000 7f8ca7fff640 10 monclient(hunting): handle_auth_done global_id 4153 payload 824 2023-11-02T15:33:12.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.663+0000 7f8ca7fff640 1 -- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca81416a0 msgr2=0x7f8ca812c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:12.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.663+0000 7f8ca7fff640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca81416a0 0x7f8ca812c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.663+0000 7f8ca7fff640 1 -- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca813e780 msgr2=0x7f8ca8131ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:12.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca813e780 0x7f8ca8131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:12.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 10 monclient: _finish_hunting 0 2023-11-02T15:33:12.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 1 monclient: found mon.noname-b 2023-11-02T15:33:12.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:12.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:12.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.664+0000 7f8ca7fff640 1 -- 172.21.15.174:0/775431227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980094d0 con 0x7f8ca80b41c0 2023-11-02T15:33:12.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.667+0000 7f8ca7fff640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009870 tx=0x7f8c980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:12.689 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.667+0000 7f8ca4ff9640 10 client.?.objecter ms_handle_connect 0x7f8ca80b41c0 2023-11-02T15:33:12.689 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.667+0000 7f8ca4ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:12.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98033d60 con 0x7f8ca80b41c0 2023-11-02T15:33:12.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:12.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:12.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 10 monclient: dump: 2023-11-02T15:33:12.692 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:12.693 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:12.693 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.693 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.694 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:12.694 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.694 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:12.694 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:12.695 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:12.695 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.695 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 10 monclient: _finish_auth 0 2023-11-02T15:33:12.695 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:33:12.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8ca4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:12.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8cae6f7640 5 monclient: authenticate success, global_id 4153 2023-11-02T15:33:12.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8cae6f7640 10 monclient: _renew_subs 2023-11-02T15:33:12.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8cae6f7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:12.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.672+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca8132fb0 con 0x7f8ca80b41c0 2023-11-02T15:33:12.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8cae6f7640 10 client.4153.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:12.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8cae6f7640 10 monclient: _renew_subs 2023-11-02T15:33:12.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8cae6f7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:12.698 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca8141340 con 0x7f8ca80b41c0 2023-11-02T15:33:12.698 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8cae6f7640 1 librados: init done 2023-11-02T15:33:12.698 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9803c400 con 0x7f8ca80b41c0 2023-11-02T15:33:12.699 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:12.699 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9803c7b0 con 0x7f8ca80b41c0 2023-11-02T15:33:12.699 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:12.700 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:12.700 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca4ff9640 10 monclient: dump: 2023-11-02T15:33:12.700 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:12.701 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:12.701 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.701 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:12.702 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:12.702 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:12.703 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:12.703 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:12.703 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:12.704 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:12.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.673+0000 7f8ca67fc640 4 set_mon_vals no callback set 2023-11-02T15:33:12.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.674+0000 7f8cae6f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:12.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.674+0000 7f8cae6f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:12.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.674+0000 7f8cae6f7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:12.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.674+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c88005040 con 0x7f8ca80b41c0 2023-11-02T15:33:12.709 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.707+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 3) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8c9803a040 con 0x7f8ca80b41c0 2023-11-02T15:33:12.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.708+0000 7f8ca4ff9640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 0x7f8c9406e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:12.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.708+0000 7f8ca77fe640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 0x7f8c9406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:12.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.708+0000 7f8ca77fe640 10 monclient: get_auth_request con 0x7f8c9406bc90 auth_method 0 2023-11-02T15:33:12.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.709+0000 7f8ca77fe640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 0x7f8c9406e150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006fd0 tx=0x7f8c9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:12.712 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.709+0000 7f8ca4ff9640 10 client.4153.objecter ms_handle_connect 0x7f8c9406bc90 2023-11-02T15:33:12.713 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.711+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8c98079b20 con 0x7f8ca80b41c0 2023-11-02T15:33:12.713 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.711+0000 7f8ca4ff9640 10 client.4153.objecter ms_dispatch 0x7f8ca8000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:12.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.711+0000 7f8ca4ff9640 3 client.4153.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:12.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.711+0000 7f8ca4ff9640 3 client.4153.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:12.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:12.711+0000 7f8ca4ff9640 20 client.4153.objecter dump_active .. 0 homeless 2023-11-02T15:33:13.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.137+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8c98075e60 con 0x7f8ca80b41c0 2023-11-02T15:33:13.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.137+0000 7f8ca4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:13.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.137+0000 7f8ca4ff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:13.481 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.479+0000 7f8cae6f7640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:13.482 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.479+0000 7f8cae6f7640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:13.482 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.479+0000 7f8cae6f7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:13.483 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.479+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c880055c0 con 0x7f8ca80b41c0 2023-11-02T15:33:13.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.559+0000 7f8ca4ff9640 1 -- 172.21.15.174:0/775431227 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8c9802f090 con 0x7f8ca80b41c0 2023-11-02T15:33:13.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.559+0000 7f8ca4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:13.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.559+0000 7f8ca4ff9640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:13.562 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:13.562 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:13.567 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 10 librados: watch_flush enter 2023-11-02T15:33:13.567 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 10 librados: watch_flush exit 2023-11-02T15:33:13.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 20 client.4153.objecter shutdown clearing up homeless session... 2023-11-02T15:33:13.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 10 client.4153.objecter successfully canceled tick 2023-11-02T15:33:13.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 msgr2=0x7f8c9406e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:13.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 0x7f8c9406e150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006fd0 tx=0x7f8c9c008040 comp rx=0 tx=0).stop 2023-11-02T15:33:13.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 10 monclient: shutdown 2023-11-02T15:33:13.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:13.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 msgr2=0x7f8ca80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:13.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009870 tx=0x7f8c980046c0 comp rx=0 tx=0).stop 2023-11-02T15:33:13.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 shutdown_connections 2023-11-02T15:33:13.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c9406bc90 0x7f8c9406e150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:13.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ca81416a0 0x7f8ca812c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:13.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca813e780 0x7f8ca8131ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:13.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 --2- 172.21.15.174:0/775431227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca80b41c0 0x7f8ca80b4e20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:13.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 >> 172.21.15.174:0/775431227 conn(0x7f8ca8065e10 msgr2=0x7f8ca8136170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:13.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 shutdown_connections 2023-11-02T15:33:13.571 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 -- 172.21.15.174:0/775431227 wait complete. 2023-11-02T15:33:13.571 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:13.565+0000 7f8cae6f7640 1 librados: shutdown 2023-11-02T15:33:13.589 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-02T15:33:18.707 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:07.967 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:18.713 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:08.004 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:18.769 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:08.029 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:18.779 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:08.070 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:18.868 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:08.163 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:18.930 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:08.225 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.034 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:08.321 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.047 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:08.323 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.096 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:08.383 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.105 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:08.392 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.109 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:08.398 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.109 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:08.386 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.138 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:08.425 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.166 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:08.453 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.181 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:08.470 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.197 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:08.484 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.325 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:33:19.318+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.340 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:33:19.333+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.513 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:33:19.503+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.638 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:33:19.631+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.665 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:33:19.657+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.729 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:33:19.721+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.729 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:33:19.721+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.756 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:33:19.749+0000 5dc2340 -1 Falling back to public interface 2023-11-02T15:33:19.763 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:09.023 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.787 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:09.078 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:19.979 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:09.275 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.096 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:09.383 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.106 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:09.382 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.118 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:09.378 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.149 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:09.440 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.181 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:09.468 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.212 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:09.501 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.234 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:09.521 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.339 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:09.635 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.396 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:20.448 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:09.735 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.461 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:09.738 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.477 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:09.737 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.484 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.482+0000 7f2c61195640 10 monclient: get_monmap_and_config 2023-11-02T15:33:20.484 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.482+0000 7f2c61195640 10 monclient: build_initial_monmap 2023-11-02T15:33:20.484 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.482+0000 7f2c61195640 10 monclient: monmap: 2023-11-02T15:33:20.485 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:20.485 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:20.485 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:20.483761+0000 2023-11-02T15:33:20.485 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:20.483761+0000 2023-11-02T15:33:20.485 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:20.486 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.486 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:20.486 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:20.486 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:20.486 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:20.487 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:20.487 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.490 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.483+0000 7f2c61195640 1 Processor -- start 2023-11-02T15:33:20.491 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.483+0000 7f2c61195640 1 -- start start 2023-11-02T15:33:20.491 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.483+0000 7f2c61195640 10 monclient: init 2023-11-02T15:33:20.491 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:20.491 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2023-11-02T15:33:20.492 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 0x7f2c5c13e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.492 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-c con 0x7f2c5c0ae3a0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:20.492 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2c5c0b3600 0x7f2c5c13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.492 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5c0b3600 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:20.492 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.493 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-b con 0x7f2c5c0ad710 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:20.493 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.493 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0ae780 con 0x7f2c5c0b3600 2023-11-02T15:33:20.493 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.493 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0add50 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.494 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.494 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0aef90 con 0x7f2c5c0ad710 2023-11-02T15:33:20.494 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:20.494 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c61195640 10 monclient(hunting): authenticate will time out at 1327.825887s 2023-11-02T15:33:20.494 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c5ad9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 0x7f2c5c13e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.495 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c5b59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.495 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.484+0000 7f2c5a59c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2c5c0b3600 0x7f2c5c13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.495 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 0x7f2c5c13e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:47092/0 (socket says 172.21.15.174:47092) 2023-11-02T15:33:20.495 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 1 -- 172.21.15.174:0/3025522132 learned_addr learned my addr 172.21.15.174:0/3025522132 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:20.495 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): get_auth_request con 0x7f2c5c0ae3a0 auth_method 0 2023-11-02T15:33:20.496 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.496 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.496 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.496 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): get_auth_request con 0x7f2c5c0b3600 auth_method 0 2023-11-02T15:33:20.496 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.497 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.497 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.497 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5b59e640 10 monclient(hunting): get_auth_request con 0x7f2c5c0ad710 auth_method 0 2023-11-02T15:33:20.497 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.497 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5b59e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.498 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5b59e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.498 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.498 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.498 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.498 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.499 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.499 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.485+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.499 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.486+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.499 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.486+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.499 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.486+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.500 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_done global_id 4194 payload 274 2023-11-02T15:33:20.500 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 -- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 msgr2=0x7f2c5c1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:20.500 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.500 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 -- 172.21.15.174:0/3025522132 >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 msgr2=0x7f2c5c13e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:20.500 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/3025522132 >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 0x7f2c5c13e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.501 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 10 monclient: _finish_hunting 0 2023-11-02T15:33:20.501 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 monclient: found mon.noname-a 2023-11-02T15:33:20.502 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:20.502 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.503 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.488+0000 7f2c5a59c640 1 -- 172.21.15.174:0/3025522132 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0af100 con 0x7f2c5c0b3600 2023-11-02T15:33:20.510 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.490+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c13ee70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0b0060 tx=0x7f2c5002b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522efe12cd9a8564 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:20.510 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 1 -- 172.21.15.174:0/3025522132 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50033640 con 0x7f2c5c0b3600 2023-11-02T15:33:20.510 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:20.511 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:20.511 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: dump: 2023-11-02T15:33:20.511 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:20.511 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:20.511 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.514 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.514 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:20.514 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.515 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:20.515 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:20.515 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:20.516 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.516 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: _finish_auth 0 2023-11-02T15:33:20.516 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: _check_auth_tickets 2023-11-02T15:33:20.517 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:20.517 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 1 -- 172.21.15.174:0/3025522132 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c500337a0 con 0x7f2c5c0b3600 2023-11-02T15:33:20.518 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:20.518 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 1 -- 172.21.15.174:0/3025522132 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50033b30 con 0x7f2c5c0b3600 2023-11-02T15:33:20.519 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:20.519 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:20.519 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c59d9b640 10 monclient: dump: 2023-11-02T15:33:20.519 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:20.520 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:20.520 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.520 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.520 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:20.520 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.521 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:20.521 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:20.521 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:20.521 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.521 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c61195640 5 monclient: authenticate success, global_id 4194 2023-11-02T15:33:20.522 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c61195640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:20.522 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.494+0000 7f2c61195640 4 set_mon_vals no callback set 2023-11-02T15:33:20.522 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 10 monclient: shutdown 2023-11-02T15:33:20.522 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:20.522 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 -- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0b3600 msgr2=0x7f2c5c13ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 --2- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c13ee70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0b0060 tx=0x7f2c5002b720 comp rx=0 tx=0).stop 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 -- 172.21.15.174:0/3025522132 shutdown_connections 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 --2- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 --2- 172.21.15.174:0/3025522132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c13ee70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 --2- 172.21.15.174:0/3025522132 >> v2:172.21.15.175:3300/0 conn(0x7f2c5c0ae3a0 0x7f2c5c13e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.523 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.495+0000 7f2c61195640 1 -- 172.21.15.174:0/3025522132 >> 172.21.15.174:0/3025522132 conn(0x7f2c5c0b7060 msgr2=0x7f2c5c0b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:20.524 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.496+0000 7f2c61195640 1 -- 172.21.15.174:0/3025522132 shutdown_connections 2023-11-02T15:33:20.524 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.496+0000 7f2c61195640 1 -- 172.21.15.174:0/3025522132 wait complete. 2023-11-02T15:33:20.524 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 10 monclient: build_initial_monmap 2023-11-02T15:33:20.524 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 10 monclient: monmap: 2023-11-02T15:33:20.524 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:20.525 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:20.525 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:20.525 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:20.525 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:20.525 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.526 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:20.526 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:20.526 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:20.526 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.526 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 librados: starting msgr at 2023-11-02T15:33:20.527 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 librados: starting objecter 2023-11-02T15:33:20.527 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 Processor -- start 2023-11-02T15:33:20.527 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 -- start start 2023-11-02T15:33:20.527 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 librados: setting wanted keys 2023-11-02T15:33:20.527 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 1 librados: calling monclient init 2023-11-02T15:33:20.528 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.497+0000 7f2c61195640 10 monclient: init 2023-11-02T15:33:20.528 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:20.528 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:33:20.528 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.528 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-c con 0x7f2c5c0ad710 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:20.529 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.529 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5c0ae3a0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:20.529 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c1d9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.529 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): picked mon.noname-b con 0x7f2c5c0b3600 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:20.529 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.530 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c142e50 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.530 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.530 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c144610 con 0x7f2c5c0b3600 2023-11-02T15:33:20.530 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:20.530 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1448c0 con 0x7f2c5c0ad710 2023-11-02T15:33:20.531 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:20.531 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.498+0000 7f2c61195640 10 monclient(hunting): authenticate will time out at 1327.840585s 2023-11-02T15:33:20.531 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.531 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5ad9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.531 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5b59e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c1d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.532 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:33218/0 (socket says 172.21.15.174:33218) 2023-11-02T15:33:20.532 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 1 -- 172.21.15.174:0/624488214 learned_addr learned my addr 172.21.15.174:0/624488214 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:20.532 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 10 monclient(hunting): get_auth_request con 0x7f2c5c0ae3a0 auth_method 0 2023-11-02T15:33:20.532 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.532 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.533 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5a59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.533 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5b59e640 10 monclient(hunting): get_auth_request con 0x7f2c5c0b3600 auth_method 0 2023-11-02T15:33:20.533 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.533 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5b59e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.533 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.499+0000 7f2c5b59e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.534 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): get_auth_request con 0x7f2c5c0ad710 auth_method 0 2023-11-02T15:33:20.534 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:20.534 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:20.534 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:20.534 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.535 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:20.536 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:20.536 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.500+0000 7f2c5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:20.536 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.502+0000 7f2c5a59c640 10 monclient(hunting): handle_auth_done global_id 4197 payload 824 2023-11-02T15:33:20.536 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 -- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 msgr2=0x7f2c5c143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:20.536 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 -- 172.21.15.174:0/624488214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c5c0b3600 msgr2=0x7f2c5c1d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c1d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 10 monclient: _finish_hunting 0 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 monclient: found mon.noname-a 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:20.537 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.538 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.503+0000 7f2c5a59c640 1 -- 172.21.15.174:0/624488214 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009400 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.538 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.505+0000 7f2c5a59c640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009890 tx=0x7f2c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:20.538 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.505+0000 7f2c437fe640 10 client.?.objecter ms_handle_connect 0x7f2c5c0ae3a0 2023-11-02T15:33:20.538 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.505+0000 7f2c437fe640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:20.539 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:09.818 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.543 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50030980 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.543 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:20.543 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:20.543 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: dump: 2023-11-02T15:33:20.543 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:20.544 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:20.544 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.544 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.544 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:20.544 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.545 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:20.545 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:20.545 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:20.545 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.545 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50030b00 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 5 monclient: authenticate success, global_id 4197 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 10 monclient: _renew_subs 2023-11-02T15:33:20.546 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.547 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c144e40 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.547 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:20.547 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50030eb0 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.547 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:20.547 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:20.548 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c437fe640 10 monclient: dump: 2023-11-02T15:33:20.548 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:20.548 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:20.548 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.548 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:20.549 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:20.549 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:20.549 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:20.549 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:20.549 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c5959a640 4 set_mon_vals no callback set 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 10 client.4197.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 10 monclient: _renew_subs 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.550 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c1d9960 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.551 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.509+0000 7f2c61195640 1 librados: init done 2023-11-02T15:33:20.551 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.511+0000 7f2c61195640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:20.551 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.511+0000 7f2c61195640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:20.551 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.511+0000 7f2c61195640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.551 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.511+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c144a50 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.552 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.532+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f2c5003a040 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.552 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.533+0000 7f2c437fe640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 0x7f2c4806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:20.552 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.533+0000 7f2c5ad9d640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 0x7f2c4806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:20.552 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.533+0000 7f2c5ad9d640 10 monclient: get_auth_request con 0x7f2c4806bd30 auth_method 0 2023-11-02T15:33:20.553 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.533+0000 7f2c5ad9d640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 0x7f2c4806e1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009bd0 tx=0x7f2c4c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:20.553 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.533+0000 7f2c437fe640 10 client.4197.objecter ms_handle_connect 0x7f2c4806bd30 2023-11-02T15:33:20.553 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.534+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2c50079910 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.553 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.534+0000 7f2c437fe640 10 client.4197.objecter ms_dispatch 0x7f2c5c000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:20.553 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.534+0000 7f2c437fe640 3 client.4197.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:20.554 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.534+0000 7f2c437fe640 3 client.4197.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:20.554 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.534+0000 7f2c437fe640 20 client.4197.objecter dump_active .. 0 homeless 2023-11-02T15:33:20.554 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:09.824 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.566 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:09.855 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.584 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:09.871 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.698 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:09.993 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.767 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.766+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2c50075c50 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.767 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.766+0000 7f2c437fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:20.768 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.766+0000 7f2c437fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:20.775 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:10.035 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.817 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:10.108 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.819 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:10.106 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.848 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:10.125 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.900 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:10.187 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.932 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:10.221 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.941 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:10.228 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.943 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:10.230 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:20.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.959+0000 7f2c61195640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:20.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.959+0000 7f2c61195640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:20.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.959+0000 7f2c61195640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:20.962 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:20.959+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c5c04e040 con 0x7f2c5c0ae3a0 2023-11-02T15:33:20.985 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:10.280 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.004 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.002+0000 7f2c437fe640 1 -- 172.21.15.174:0/624488214 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2c50038020 con 0x7f2c5c0ae3a0 2023-11-02T15:33:21.004 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.002+0000 7f2c437fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:21.005 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.002+0000 7f2c437fe640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:21.005 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:21.005 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:21.007 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 10 librados: watch_flush enter 2023-11-02T15:33:21.007 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 10 librados: watch_flush exit 2023-11-02T15:33:21.008 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 20 client.4197.objecter shutdown clearing up homeless session... 2023-11-02T15:33:21.008 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 10 client.4197.objecter successfully canceled tick 2023-11-02T15:33:21.008 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 msgr2=0x7f2c4806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:21.008 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 0x7f2c4806e1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009bd0 tx=0x7f2c4c009240 comp rx=0 tx=0).stop 2023-11-02T15:33:21.008 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 10 monclient: shutdown 2023-11-02T15:33:21.009 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:21.009 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 msgr2=0x7f2c5c1440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:21.009 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009890 tx=0x7f2c500046c0 comp rx=0 tx=0).stop 2023-11-02T15:33:21.009 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 shutdown_connections 2023-11-02T15:33:21.009 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c5c0b3600 0x7f2c5c1d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:21.010 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c5c0ae3a0 0x7f2c5c1440d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:21.010 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2c4806bd30 0x7f2c4806e1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:21.010 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 --2- 172.21.15.174:0/624488214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5c0ad710 0x7f2c5c143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:21.010 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 >> 172.21.15.174:0/624488214 conn(0x7f2c5c0b7060 msgr2=0x7f2c5c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:21.011 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 shutdown_connections 2023-11-02T15:33:21.011 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 -- 172.21.15.174:0/624488214 wait complete. 2023-11-02T15:33:21.011 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:21.005+0000 7f2c61195640 1 librados: shutdown 2023-11-02T15:33:21.016 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-02T15:33:21.068 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:10.328 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.108 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:10.398 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.118 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:10.405 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.149 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:10.426 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.223 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:10.512 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.228 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:10.515 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.250 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:10.537 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.277 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:10.572 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.360 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:10.620 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.402 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:10.693 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.416 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:10.703 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.441 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:10.718 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.517 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:10.806 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.517 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:10.804 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.547 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:10.834 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.564 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:10.859 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.647 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:10.907 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.691 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:10.982 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.704 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:10.991 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.746 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:11.023 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.804 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:11.093 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.805 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:11.092 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.833 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:11.120 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.852 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:11.147 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.936 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:11.196 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.981 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:11.272 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:21.994 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:11.281 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.041 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:11.318 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.094 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:11.383 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.096 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:11.383 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.120 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:11.407 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.143 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:11.438 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.228 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:11.488 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.271 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:11.562 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.282 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:11.569 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.330 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:11.607 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.381 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:11.671 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.384 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:11.671 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.410 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:11.697 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.435 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:11.730 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.572 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:11.859 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.615 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:11.892 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.676 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:11.965 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.678 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:11.965 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.761 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:12.021 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.816 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:12.076 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.825 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:12.116 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.880 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:12.170 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.927 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:12.214 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:22.985 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:12.280 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.008 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:12.295 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.040 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:12.335 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.139 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:12.415 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.142 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:12.429 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.195 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:12.471 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.243 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:12.530 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.265 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:12.554 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.267 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:12.554 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.321 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:12.609 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:23.340 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:12.629 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:27.824 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:27.917 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.916+0000 7f0c8b2c4640 10 monclient: get_monmap_and_config 2023-11-02T15:33:27.918 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.916+0000 7f0c8b2c4640 10 monclient: build_initial_monmap 2023-11-02T15:33:27.918 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.916+0000 7f0c8b2c4640 10 monclient: monmap: 2023-11-02T15:33:27.918 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:27.918 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:27.919 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:27.917324+0000 2023-11-02T15:33:27.919 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:27.917324+0000 2023-11-02T15:33:27.919 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:27.919 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.919 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:27.920 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:27.920 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:27.920 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:27.920 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:27.920 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.922 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 1 Processor -- start 2023-11-02T15:33:27.922 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 1 -- start start 2023-11-02T15:33:27.922 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient: init 2023-11-02T15:33:27.923 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:27.923 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient: _add_conns ranks=[0,1,2,3,4] 2023-11-02T15:33:27.923 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c840ae3a0 addr v1:172.21.15.174:6789/0 2023-11-02T15:33:27.923 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0c840b3600 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:27.924 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 0x7f0c84141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:27.924 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-b con 0x7f0c8413f040 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:27.924 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.917+0000 7f0c8b2c4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c840add50 con 0x7f0c840ae3a0 2023-11-02T15:33:27.924 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c8b2c4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c84142510 con 0x7f0c840b3600 2023-11-02T15:33:27.925 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c8b2c4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:27.925 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c8b2c4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840ae780 con 0x7f0c8413f040 2023-11-02T15:33:27.925 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c8b2c4640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:27.925 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c8b2c4640 10 monclient(hunting): authenticate will time out at 1335.259702s 2023-11-02T15:33:27.926 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c89060640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f0c840ae3a0 0x7f0c8413e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:55920/0 (socket says 172.21.15.174:55920) 2023-11-02T15:33:27.926 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.918+0000 7f0c89060640 1 -- 172.21.15.174:0/2940141437 learned_addr learned my addr 172.21.15.174:0/2940141437 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:33:27.926 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 1 --2- 172.21.15.174:0/2940141437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 0x7f0c84141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:27.926 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): get_auth_request con 0x7f0c8413f040 auth_method 0 2023-11-02T15:33:27.926 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:27.927 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.927 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.927 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:27.927 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:27.928 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.919+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:27.928 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381942304 0 0) 0x7f0c84142510 con 0x7f0c840b3600 2023-11-02T15:33:27.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _negotiate 2023-11-02T15:33:27.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): my global_id is 4189 2023-11-02T15:33:27.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c70003520 con 0x7f0c840b3600 2023-11-02T15:33:27.930 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940533417 0 0) 0x7f0c840add50 con 0x7f0c840ae3a0 2023-11-02T15:33:27.930 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _negotiate 2023-11-02T15:33:27.930 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.931 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.931 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 10 monclient(hunting): my global_id is 4200 2023-11-02T15:33:27.931 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.921+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c84142510 con 0x7f0c840ae3a0 2023-11-02T15:33:27.931 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 10 monclient(hunting): handle_auth_done global_id 4178 payload 274 2023-11-02T15:33:27.931 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 1 -- 172.21.15.174:0/2940141437 >> v1:172.21.15.175:6789/0 conn(0x7f0c840b3600 legacy=0x7f0c841418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 1 -- 172.21.15.174:0/2940141437 >> v1:172.21.15.174:6789/0 conn(0x7f0c840ae3a0 legacy=0x7f0c8413e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 10 monclient: _finish_hunting 0 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 1 monclient: found mon.noname-b 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:33:27.932 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.922+0000 7f0c89861640 1 -- 172.21.15.174:0/2940141437 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840aef90 con 0x7f0c8413f040 2023-11-02T15:33:27.933 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.924+0000 7f0c89861640 1 --2- 172.21.15.174:0/2940141437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 0x7f0c84141fd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009dc0 tx=0x7f0c7c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde36b4943a66fb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:27.936 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c02cc40 con 0x7f0c8413f040 2023-11-02T15:33:27.936 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:27.936 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:27.936 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: dump: 2023-11-02T15:33:27.936 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:27.937 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:27.937 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.937 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.937 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:27.937 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.938 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:27.938 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:27.938 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:27.938 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.938 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: _finish_auth 0 2023-11-02T15:33:27.939 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: _check_auth_tickets 2023-11-02T15:33:27.939 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:27.939 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c02cda0 con 0x7f0c8413f040 2023-11-02T15:33:27.939 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:27.939 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 1 -- 172.21.15.174:0/2940141437 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c034660 con 0x7f0c8413f040 2023-11-02T15:33:27.940 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:27.940 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:27.940 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c7bfff640 10 monclient: dump: 2023-11-02T15:33:27.940 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:27.940 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:27.941 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.941 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.941 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:27.941 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.941 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:27.942 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:27.942 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:27.942 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.942 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 5 monclient: authenticate success, global_id 4178 2023-11-02T15:33:27.942 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:27.943 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 4 set_mon_vals no callback set 2023-11-02T15:33:27.943 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 10 monclient: shutdown 2023-11-02T15:33:27.943 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:27.943 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/2940141437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 msgr2=0x7f0c84141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:27.944 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/2940141437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 0x7f0c84141fd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009dc0 tx=0x7f0c7c02be90 comp rx=0 tx=0).stop 2023-11-02T15:33:27.944 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/2940141437 shutdown_connections 2023-11-02T15:33:27.944 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/2940141437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c8413f040 0x7f0c84141fd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:27.944 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/2940141437 >> 172.21.15.174:0/2940141437 conn(0x7f0c840b7060 msgr2=0x7f0c840b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:27.944 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.929+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/2940141437 shutdown_connections 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.930+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/2940141437 wait complete. 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.930+0000 7f0c8b2c4640 10 monclient: build_initial_monmap 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.930+0000 7f0c8b2c4640 10 monclient: monmap: 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:27.945 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:27.946 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:27.946 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:27.946 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.946 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:27.947 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:27.947 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:27.947 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.947 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.930+0000 7f0c8b2c4640 1 librados: starting msgr at 2023-11-02T15:33:27.948 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.930+0000 7f0c8b2c4640 1 librados: starting objecter 2023-11-02T15:33:27.948 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 Processor -- start 2023-11-02T15:33:27.948 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 -- start start 2023-11-02T15:33:27.948 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 librados: setting wanted keys 2023-11-02T15:33:27.948 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 librados: calling monclient init 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient: init 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 0x7f0c8413a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-b con 0x7f0c840ae3a0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:27.949 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:27.950 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-c con 0x7f0c840b3600 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:27.950 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c8413f040 0x7f0c84136380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:27.950 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): picked mon.noname-a con 0x7f0c8413f040 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:27.950 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:27.950 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84137b90 con 0x7f0c8413f040 2023-11-02T15:33:27.951 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:27.951 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8413ade0 con 0x7f0c840ae3a0 2023-11-02T15:33:27.951 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:27.951 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840a8640 con 0x7f0c840b3600 2023-11-02T15:33:27.951 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:27.952 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.931+0000 7f0c8b2c4640 10 monclient(hunting): authenticate will time out at 1335.273436s 2023-11-02T15:33:27.952 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:27.952 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 0x7f0c8413a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:27.952 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89861640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c8413f040 0x7f0c84136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:27.952 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:36606/0 (socket says 172.21.15.174:36606) 2023-11-02T15:33:27.953 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 0x7f0c8413a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:40162/0 (socket says 172.21.15.174:40162) 2023-11-02T15:33:27.953 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 1 -- 172.21.15.174:0/562994436 learned_addr learned my addr 172.21.15.174:0/562994436 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:27.953 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 10 monclient(hunting): get_auth_request con 0x7f0c840b3600 auth_method 0 2023-11-02T15:33:27.953 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:27.953 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.954 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c8885f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.954 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 10 monclient(hunting): get_auth_request con 0x7f0c840ae3a0 auth_method 0 2023-11-02T15:33:27.954 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:27.954 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.954 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89060640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89861640 10 monclient(hunting): get_auth_request con 0x7f0c8413f040 auth_method 0 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89861640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.932+0000 7f0c89861640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c8885f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:27.955 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c8885f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:27.956 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c8885f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:27.956 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89060640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:27.956 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89060640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:27.956 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89060640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:27.957 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:27.957 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:27.957 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.933+0000 7f0c89861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:27.959 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.935+0000 7f0c8885f640 10 monclient(hunting): handle_auth_done global_id 4181 payload 824 2023-11-02T15:33:27.959 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 -- 172.21.15.174:0/562994436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 msgr2=0x7f0c8413a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:27.959 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 0x7f0c8413a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:27.960 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 -- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c8413f040 msgr2=0x7f0c84136380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:27.960 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c8413f040 0x7f0c84136380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:27.960 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 10 monclient: _finish_hunting 0 2023-11-02T15:33:27.960 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 monclient: found mon.noname-c 2023-11-02T15:33:27.960 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:27.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:33:27.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.936+0000 7f0c8885f640 1 -- 172.21.15.174:0/562994436 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c009a70 con 0x7f0c840b3600 2023-11-02T15:33:27.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.938+0000 7f0c8885f640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0c74001eb0 tx=0x7f0c7400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:27.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.938+0000 7f0c79ffb640 10 client.?.objecter ms_handle_connect 0x7f0c840b3600 2023-11-02T15:33:27.961 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.938+0000 7f0c79ffb640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:27.962 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740052d0 con 0x7f0c840b3600 2023-11-02T15:33:27.962 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:27.962 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:27.962 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: dump: 2023-11-02T15:33:27.962 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:27.963 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:27.963 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.963 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.963 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:27.963 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.964 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:27.964 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:27.964 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: _finish_auth 0 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74005450 con 0x7f0c840b3600 2023-11-02T15:33:27.965 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:27.966 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74012870 con 0x7f0c840b3600 2023-11-02T15:33:27.966 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:27.966 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:27.966 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c79ffb640 10 monclient: dump: 2023-11-02T15:33:27.966 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:27.967 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:27.967 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.967 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:27.967 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:27.967 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:27.968 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:27.968 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:27.968 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:27.968 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:27.968 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c7b7fe640 4 set_mon_vals no callback set 2023-11-02T15:33:27.969 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 5 monclient: authenticate success, global_id 4181 2023-11-02T15:33:27.969 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 10 monclient: _renew_subs 2023-11-02T15:33:27.969 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:27.969 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c841368c0 con 0x7f0c840b3600 2023-11-02T15:33:27.969 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 10 client.4181.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:27.970 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 10 monclient: _renew_subs 2023-11-02T15:33:27.970 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:27.970 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c84136d80 con 0x7f0c840b3600 2023-11-02T15:33:27.970 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.942+0000 7f0c8b2c4640 1 librados: init done 2023-11-02T15:33:27.970 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.943+0000 7f0c8b2c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:27.971 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.943+0000 7f0c8b2c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:27.971 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.943+0000 7f0c8b2c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:27.971 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:27.943+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8404e040 con 0x7f0c840b3600 2023-11-02T15:33:28.211 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:17.471 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:28.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.318+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f0c740035f0 con 0x7f0c840b3600 2023-11-02T15:33:28.321 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.319+0000 7f0c79ffb640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 0x7f0c6806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:28.321 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.319+0000 7f0c89060640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 0x7f0c6806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:28.321 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.319+0000 7f0c89060640 10 monclient: get_auth_request con 0x7f0c6806bd30 auth_method 0 2023-11-02T15:33:28.322 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.320+0000 7f0c89060640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 0x7f0c6806e1f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0c80000f60 tx=0x7f0c80007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:28.322 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.320+0000 7f0c79ffb640 10 client.4181.objecter ms_handle_connect 0x7f0c6806bd30 2023-11-02T15:33:28.340 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0c74057420 con 0x7f0c840b3600 2023-11-02T15:33:28.340 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 10 client.4181.objecter ms_dispatch 0x7f0c84000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:28.340 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 3 client.4181.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:28.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 3 client.4181.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:28.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 20 client.4181.objecter dump_active .. 0 homeless 2023-11-02T15:33:28.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0c74053760 con 0x7f0c840b3600 2023-11-02T15:33:28.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:28.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.338+0000 7f0c79ffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:28.522 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:17.809 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:28.526 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:17.813 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:28.554 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:17.849 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:28.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.618+0000 7f0c8b2c4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:28.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.618+0000 7f0c8b2c4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:28.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.618+0000 7f0c8b2c4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:28.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.618+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c840afe40 con 0x7f0c840b3600 2023-11-02T15:33:28.648 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.646+0000 7f0c79ffb640 1 -- 172.21.15.174:0/562994436 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0c74017090 con 0x7f0c840b3600 2023-11-02T15:33:28.649 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.646+0000 7f0c79ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:28.649 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.646+0000 7f0c79ffb640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:28.650 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:28.650 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:28.653 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 10 librados: watch_flush enter 2023-11-02T15:33:28.653 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 10 librados: watch_flush exit 2023-11-02T15:33:28.654 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 20 client.4181.objecter shutdown clearing up homeless session... 2023-11-02T15:33:28.654 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 10 client.4181.objecter successfully canceled tick 2023-11-02T15:33:28.654 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 msgr2=0x7f0c6806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:28.655 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 0x7f0c6806e1f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0c80000f60 tx=0x7f0c80007d30 comp rx=0 tx=0).stop 2023-11-02T15:33:28.655 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 10 monclient: shutdown 2023-11-02T15:33:28.655 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:28.656 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 msgr2=0x7f0c84135810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:28.656 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0c74001eb0 tx=0x7f0c7400c960 comp rx=0 tx=0).stop 2023-11-02T15:33:28.656 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 shutdown_connections 2023-11-02T15:33:28.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c8413f040 0x7f0c84136380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:28.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f0c6806bd30 0x7f0c6806e1f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:28.657 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c840b3600 0x7f0c84135810 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:28.658 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 --2- 172.21.15.174:0/562994436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c840ae3a0 0x7f0c8413a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:28.658 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.649+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 >> 172.21.15.174:0/562994436 conn(0x7f0c840b7060 msgr2=0x7f0c841418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:28.658 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.650+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 shutdown_connections 2023-11-02T15:33:28.659 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.650+0000 7f0c8b2c4640 1 -- 172.21.15.174:0/562994436 wait complete. 2023-11-02T15:33:28.659 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:28.650+0000 7f0c8b2c4640 1 librados: shutdown 2023-11-02T15:33:28.662 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-02T15:33:28.673 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:17.964 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:29.254 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:18.541 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:29.307 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:18.584 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:29.424 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:18.714 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:32.796 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:22.056 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:32.802 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:22.063 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.578 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:22.864 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.582 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:22.868 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.735 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:23.031 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.740 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:23.035 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.862 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:23.149 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:33.866 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:23.153 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.036 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:23.323 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.037 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:23.327 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.660 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:23.947 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.662 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:23.951 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.692 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:23.978 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.696 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:23.972 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.697 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:23.984 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:34.702 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:23.979 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:35.471 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:35.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 10 monclient: get_monmap_and_config 2023-11-02T15:33:35.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 10 monclient: build_initial_monmap 2023-11-02T15:33:35.561 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 10 monclient: monmap: 2023-11-02T15:33:35.561 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:35.562 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:35.562 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:35.559969+0000 2023-11-02T15:33:35.562 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:35.559969+0000 2023-11-02T15:33:35.562 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:35.562 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.563 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:35.563 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:35.563 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:35.563 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:35.563 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:35.564 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.564 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 1 Processor -- start 2023-11-02T15:33:35.564 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 1 -- start start 2023-11-02T15:33:35.564 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.559+0000 7f906d9dd640 10 monclient: init 2023-11-02T15:33:35.568 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:35.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2023-11-02T15:33:35.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f90680b41c0 0x7f90680a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-c con 0x7f90680b41c0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:35.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f906813e780 0x7f906813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.569 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-a con 0x7f906813e780 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:35.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90681416a0 0x7f9068141b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-b con 0x7f90681416a0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:35.570 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.572 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90680aa160 con 0x7f906813e780 2023-11-02T15:33:35.572 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.573 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90680b45a0 con 0x7f90680b41c0 2023-11-02T15:33:35.573 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.573 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9068142050 con 0x7f90681416a0 2023-11-02T15:33:35.573 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:35.574 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f906d9dd640 10 monclient(hunting): authenticate will time out at 1342.902192s 2023-11-02T15:33:35.574 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.560+0000 7f90667fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f906813e780 0x7f906813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.574 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f90680b41c0 0x7f90680a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.575 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90677fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90681416a0 0x7f9068141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.575 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f90680b41c0 0x7f90680a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:38684/0 (socket says 172.21.15.174:38684) 2023-11-02T15:33:35.575 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 1 -- 172.21.15.174:0/2898065112 learned_addr learned my addr 172.21.15.174:0/2898065112 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:35.576 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): get_auth_request con 0x7f90680b41c0 auth_method 0 2023-11-02T15:33:35.576 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.576 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.576 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.576 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90667fc640 10 monclient(hunting): get_auth_request con 0x7f906813e780 auth_method 0 2023-11-02T15:33:35.577 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.578 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90667fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.578 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90667fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.578 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90677fe640 10 monclient(hunting): get_auth_request con 0x7f90681416a0 auth_method 0 2023-11-02T15:33:35.578 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.578 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90677fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.579 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f90677fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.579 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.579 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.579 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.561+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.580 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.580 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.580 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.580 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.581 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.581 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.562+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.581 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 10 monclient(hunting): handle_auth_done global_id 4195 payload 274 2023-11-02T15:33:35.581 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 -- 172.21.15.174:0/2898065112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90681416a0 msgr2=0x7f9068141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:35.581 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 --2- 172.21.15.174:0/2898065112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90681416a0 0x7f9068141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 -- 172.21.15.174:0/2898065112 >> v2:172.21.15.174:3300/0 conn(0x7f906813e780 msgr2=0x7f906813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:35.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 --2- 172.21.15.174:0/2898065112 >> v2:172.21.15.174:3300/0 conn(0x7f906813e780 0x7f906813ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 10 monclient: _finish_hunting 0 2023-11-02T15:33:35.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 monclient: found mon.noname-c 2023-11-02T15:33:35.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:35.583 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:33:35.583 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.563+0000 7f9066ffd640 1 -- 172.21.15.174:0/2898065112 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680aa360 con 0x7f90680b41c0 2023-11-02T15:33:35.583 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.565+0000 7f9066ffd640 1 --2- 172.21.15.174:0/2898065112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90680b41c0 0x7f90680a8870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f90680669c0 tx=0x7f905802b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed73f04df77f2640 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:35.595 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.569+0000 7f9065ffb640 1 -- 172.21.15.174:0/2898065112 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058031750 con 0x7f90680b41c0 2023-11-02T15:33:35.595 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:35.595 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:35.596 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: dump: 2023-11-02T15:33:35.596 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:35.596 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:35.596 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.596 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.597 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:35.597 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.597 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:35.597 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:35.597 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:35.598 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.598 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: _finish_auth 0 2023-11-02T15:33:35.598 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:33:35.598 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:35.599 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 1 -- 172.21.15.174:0/2898065112 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90580318b0 con 0x7f90680b41c0 2023-11-02T15:33:35.600 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:35.600 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 1 -- 172.21.15.174:0/2898065112 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058031c40 con 0x7f90680b41c0 2023-11-02T15:33:35.600 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:35.600 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:35.601 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f9065ffb640 10 monclient: dump: 2023-11-02T15:33:35.601 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:35.601 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:35.601 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.601 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.602 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:35.602 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.602 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:35.602 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:35.603 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:35.603 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.603 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 5 monclient: authenticate success, global_id 4195 2023-11-02T15:33:35.603 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:35.604 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 4 set_mon_vals no callback set 2023-11-02T15:33:35.604 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 10 monclient: shutdown 2023-11-02T15:33:35.604 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:35.605 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 1 -- 172.21.15.174:0/2898065112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90680b41c0 msgr2=0x7f90680a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:35.605 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.570+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2898065112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90680b41c0 0x7f90680a8870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f90680669c0 tx=0x7f905802b7f0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.605 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 -- 172.21.15.174:0/2898065112 shutdown_connections 2023-11-02T15:33:35.606 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2898065112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90681416a0 0x7f9068141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.606 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2898065112 >> v2:172.21.15.174:3300/0 conn(0x7f906813e780 0x7f906813ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.606 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2898065112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90680b41c0 0x7f90680a8870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.606 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 -- 172.21.15.174:0/2898065112 >> 172.21.15.174:0/2898065112 conn(0x7f9068065e10 msgr2=0x7f9068066220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:35.606 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 -- 172.21.15.174:0/2898065112 shutdown_connections 2023-11-02T15:33:35.607 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.571+0000 7f906d9dd640 1 -- 172.21.15.174:0/2898065112 wait complete. 2023-11-02T15:33:35.607 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 10 monclient: build_initial_monmap 2023-11-02T15:33:35.607 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 10 monclient: monmap: 2023-11-02T15:33:35.607 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:35.607 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:35.608 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:35.609 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:35.609 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.609 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 librados: starting msgr at 2023-11-02T15:33:35.609 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 librados: starting objecter 2023-11-02T15:33:35.609 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 Processor -- start 2023-11-02T15:33:35.610 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 -- start start 2023-11-02T15:33:35.610 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 librados: setting wanted keys 2023-11-02T15:33:35.610 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 1 librados: calling monclient init 2023-11-02T15:33:35.610 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.572+0000 7f906d9dd640 10 monclient: init 2023-11-02T15:33:35.610 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.573+0000 7f906d9dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:35.611 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.573+0000 7f906d9dd640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:33:35.611 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.573+0000 7f906d9dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90680b41c0 0x7f9068140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.611 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.573+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-c con 0x7f90680b41c0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:35.611 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.573+0000 7f906d9dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 0x7f9068144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.611 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-a con 0x7f906813e780 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:35.612 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 0x7f90680b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.612 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 0x7f9068144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.612 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): picked mon.noname-b con 0x7f90681416a0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:35.612 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.612 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9068145670 con 0x7f906813e780 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9068144360 con 0x7f90681416a0 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90681444d0 con 0x7f90680b41c0 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:35.613 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f906d9dd640 10 monclient(hunting): authenticate will time out at 1342.916128s 2023-11-02T15:33:35.614 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f9066ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90680b41c0 0x7f9068140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.614 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 0x7f9068144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40296/0 (socket says 172.21.15.174:40296) 2023-11-02T15:33:35.614 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 1 -- 172.21.15.174:0/2397602251 learned_addr learned my addr 172.21.15.174:0/2397602251 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:35.614 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 10 monclient(hunting): get_auth_request con 0x7f906813e780 auth_method 0 2023-11-02T15:33:35.614 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.615 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.615 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90667fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.615 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90677fe640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 0x7f90680b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.615 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f9066ffd640 10 monclient(hunting): get_auth_request con 0x7f90680b41c0 auth_method 0 2023-11-02T15:33:35.615 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f9066ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.616 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f9066ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.616 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f9066ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.616 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90677fe640 10 monclient(hunting): get_auth_request con 0x7f90681416a0 auth_method 0 2023-11-02T15:33:35.616 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:35.616 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90677fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.574+0000 7f90677fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.617 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.618 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f9066ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.618 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:35.618 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:35.618 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.575+0000 7f90677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:35.618 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 10 monclient(hunting): handle_auth_done global_id 4198 payload 824 2023-11-02T15:33:35.619 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 -- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90680b41c0 msgr2=0x7f9068140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:35.619 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90680b41c0 0x7f9068140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.619 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 -- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 msgr2=0x7f9068144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:35.619 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 0x7f9068144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:35.619 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 10 monclient: _finish_hunting 0 2023-11-02T15:33:35.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 monclient: found mon.noname-b 2023-11-02T15:33:35.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:35.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:35.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.578+0000 7f90677fe640 1 -- 172.21.15.174:0/2397602251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580094d0 con 0x7f90681416a0 2023-11-02T15:33:35.620 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.580+0000 7f90677fe640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 0x7f90680b7610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f906000eba0 tx=0x7f906000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:35.621 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.580+0000 7f904ffff640 10 client.?.objecter ms_handle_connect 0x7f90681416a0 2023-11-02T15:33:35.621 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.580+0000 7f904ffff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:35.621 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906000cd20 con 0x7f90681416a0 2023-11-02T15:33:35.621 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:35.621 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 10 monclient: dump: 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.622 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:35.623 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.623 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:35.623 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:35.623 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:35.623 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.624 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 10 monclient: _finish_auth 0 2023-11-02T15:33:35.624 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 10 monclient: _check_auth_tickets 2023-11-02T15:33:35.624 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f904ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:35.624 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f906d9dd640 5 monclient: authenticate success, global_id 4198 2023-11-02T15:33:35.624 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f906d9dd640 10 monclient: _renew_subs 2023-11-02T15:33:35.625 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f906d9dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:35.625 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.582+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9068145160 con 0x7f90681416a0 2023-11-02T15:33:35.625 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.583+0000 7f906d9dd640 10 client.4198.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:35.625 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.583+0000 7f906d9dd640 10 monclient: _renew_subs 2023-11-02T15:33:35.625 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.583+0000 7f906d9dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.583+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90680b7d30 con 0x7f90681416a0 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.583+0000 7f906d9dd640 1 librados: init done 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f906d9dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f906d9dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f906d9dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:35.626 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90680b5dc0 con 0x7f90681416a0 2023-11-02T15:33:35.627 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906000f040 con 0x7f90681416a0 2023-11-02T15:33:35.627 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:35.627 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060014680 con 0x7f90681416a0 2023-11-02T15:33:35.627 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:35.627 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:35.628 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f904ffff640 10 monclient: dump: 2023-11-02T15:33:35.628 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:35.628 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:35.628 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.628 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:35.629 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:35.630 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.584+0000 7f90657fa640 4 set_mon_vals no callback set 2023-11-02T15:33:35.927 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.925+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f90600147e0 con 0x7f90681416a0 2023-11-02T15:33:35.928 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.926+0000 7f904ffff640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 0x7f905406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:35.928 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.926+0000 7f9066ffd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 0x7f905406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:35.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.926+0000 7f9066ffd640 10 monclient: get_auth_request con 0x7f905406bd30 auth_method 0 2023-11-02T15:33:35.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.927+0000 7f904ffff640 10 client.4198.objecter ms_handle_connect 0x7f905406bd30 2023-11-02T15:33:35.929 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.927+0000 7f9066ffd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 0x7f905406e1f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90680669c0 tx=0x7f9058008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:35.940 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.938+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9060058bd0 con 0x7f90681416a0 2023-11-02T15:33:35.940 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.938+0000 7f904ffff640 10 client.4198.objecter ms_dispatch 0x7f9068000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:35.941 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.938+0000 7f904ffff640 3 client.4198.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:35.941 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.938+0000 7f904ffff640 3 client.4198.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:35.941 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.939+0000 7f904ffff640 20 client.4198.objecter dump_active .. 0 homeless 2023-11-02T15:33:35.942 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.940+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9060054f10 con 0x7f90681416a0 2023-11-02T15:33:35.942 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.940+0000 7f904ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:35.942 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:35.940+0000 7f904ffff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:36.164 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.162+0000 7f906d9dd640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:36.165 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.162+0000 7f906d9dd640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:36.165 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.162+0000 7f906d9dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:36.165 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.162+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90680b5dc0 con 0x7f90681416a0 2023-11-02T15:33:36.184 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.180+0000 7f904ffff640 1 -- 172.21.15.174:0/2397602251 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f90680b5dc0 con 0x7f90681416a0 2023-11-02T15:33:36.184 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.180+0000 7f904ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:36.184 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.180+0000 7f904ffff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:36.185 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:36.185 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:36.187 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 10 librados: watch_flush enter 2023-11-02T15:33:36.188 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 10 librados: watch_flush exit 2023-11-02T15:33:36.188 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 20 client.4198.objecter shutdown clearing up homeless session... 2023-11-02T15:33:36.188 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 10 client.4198.objecter successfully canceled tick 2023-11-02T15:33:36.189 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 msgr2=0x7f905406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:36.189 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 0x7f905406e1f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90680669c0 tx=0x7f9058008030 comp rx=0 tx=0).stop 2023-11-02T15:33:36.189 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 10 monclient: shutdown 2023-11-02T15:33:36.189 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:36.190 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 msgr2=0x7f90680b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:36.190 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 0x7f90680b7610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f906000eba0 tx=0x7f906000c490 comp rx=0 tx=0).stop 2023-11-02T15:33:36.190 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 shutdown_connections 2023-11-02T15:33:36.191 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90681416a0 0x7f90680b7610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:36.191 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906813e780 0x7f9068144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:36.191 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f905406bd30 0x7f905406e1f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:36.191 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 --2- 172.21.15.174:0/2397602251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90680b41c0 0x7f9068140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:36.191 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.182+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 >> 172.21.15.174:0/2397602251 conn(0x7f9068065e10 msgr2=0x7f9068139800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:36.192 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.185+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 shutdown_connections 2023-11-02T15:33:36.192 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.185+0000 7f906d9dd640 1 -- 172.21.15.174:0/2397602251 wait complete. 2023-11-02T15:33:36.192 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:36.185+0000 7f906d9dd640 1 librados: shutdown 2023-11-02T15:33:36.197 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-02T15:33:37.015 INFO:tasks.ceph.osd.7.smithi175.stderr:==00:00:00:26.275 86418== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:37.829 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:27.116 86148== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:38.203 INFO:tasks.ceph.osd.4.smithi175.stderr:==00:00:00:27.498 86415== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:38.317 INFO:tasks.ceph.osd.6.smithi175.stderr:==00:00:00:27.605 86417== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:38.552 INFO:tasks.ceph.osd.5.smithi175.stderr:==00:00:00:27.843 86416== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:38.949 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:28.239 86146== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:39.405 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:28.681 86153== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:39.537 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:28.824 86149== Warning: unimplemented fcntl command: 1038 2023-11-02T15:33:43.005 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:43.097 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.096+0000 7f77d3cca640 10 monclient: get_monmap_and_config 2023-11-02T15:33:43.097 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.096+0000 7f77d3cca640 10 monclient: build_initial_monmap 2023-11-02T15:33:43.098 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.096+0000 7f77d3cca640 10 monclient: monmap: 2023-11-02T15:33:43.098 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:43.098 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:43.098 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:43.097020+0000 2023-11-02T15:33:43.099 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:43.097020+0000 2023-11-02T15:33:43.099 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:43.099 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.099 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:43.099 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:43.100 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:43.100 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:43.100 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:43.100 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.105 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.096+0000 7f77d3cca640 1 Processor -- start 2023-11-02T15:33:43.105 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.097+0000 7f77d3cca640 1 -- start start 2023-11-02T15:33:43.105 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.097+0000 7f77d3cca640 10 monclient: init 2023-11-02T15:33:43.105 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:43.106 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2023-11-02T15:33:43.106 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f77cc065c60 0x7f77cc13c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.106 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-a con 0x7f77cc065c60 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:43.106 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f77cc13cd80 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:43.106 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77cc13fca0 0x7f77cc149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.107 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-c con 0x7f77cc13fca0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:43.107 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc0b2fc0 con 0x7f77cc13cd80 2023-11-02T15:33:43.107 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:43.107 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc066040 con 0x7f77cc065c60 2023-11-02T15:33:43.107 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:43.108 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc071170 con 0x7f77cc13fca0 2023-11-02T15:33:43.108 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:43.108 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d3cca640 10 monclient(hunting): authenticate will time out at 1350.439964s 2023-11-02T15:33:43.108 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f77cc065c60 0x7f77cc13c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.108 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f77cc065c60 0x7f77cc13c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40304/0 (socket says 172.21.15.174:40304) 2023-11-02T15:33:43.109 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 1 -- 172.21.15.174:0/3986434264 learned_addr learned my addr 172.21.15.174:0/3986434264 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:43.109 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 10 monclient(hunting): get_auth_request con 0x7f77cc065c60 auth_method 0 2023-11-02T15:33:43.109 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:43.109 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:43.109 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.098+0000 7f77d1a66640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:43.110 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.099+0000 7f77d2267640 1 --2- 172.21.15.174:0/3986434264 >> v2:172.21.15.175:3300/0 conn(0x7f77cc13fca0 0x7f77cc149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.110 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.099+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:43.110 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.099+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:43.110 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.099+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:43.110 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.100+0000 7f77d2267640 10 monclient(hunting): get_auth_request con 0x7f77cc13fca0 auth_method 0 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.100+0000 7f77d2267640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.100+0000 7f77d2267640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.100+0000 7f77d2267640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 10 monclient(hunting): handle_auth_done global_id 4212 payload 274 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 1 -- 172.21.15.174:0/3986434264 >> v2:172.21.15.175:3300/0 conn(0x7f77cc13fca0 msgr2=0x7f77cc149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:43.111 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 1 --2- 172.21.15.174:0/3986434264 >> v2:172.21.15.175:3300/0 conn(0x7f77cc13fca0 0x7f77cc149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.112 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 1 -- 172.21.15.174:0/3986434264 >> v1:172.21.15.175:6789/0 conn(0x7f77cc13cd80 legacy=0x7f77cc13f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.112 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 10 monclient: _finish_hunting 0 2023-11-02T15:33:43.112 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 1 monclient: found mon.noname-a 2023-11-02T15:33:43.113 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:43.113 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:43.113 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.101+0000 7f77d1a66640 1 -- 172.21.15.174:0/3986434264 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc0adb70 con 0x7f77cc065c60 2023-11-02T15:33:43.113 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.103+0000 7f77d1a66640 1 --2- 172.21.15.174:0/3986434264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc065c60 0x7f77cc13c840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0addc0 tx=0x7f77c802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32421f525d24fc33 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:43.118 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 1 -- 172.21.15.174:0/3986434264 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030800 con 0x7f77cc065c60 2023-11-02T15:33:43.119 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:43.119 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:43.119 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: dump: 2023-11-02T15:33:43.119 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:43.119 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:43.120 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.120 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.120 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:43.120 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.121 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:43.121 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:43.121 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:43.121 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.122 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: _finish_auth 0 2023-11-02T15:33:43.122 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: _check_auth_tickets 2023-11-02T15:33:43.122 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:43.123 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 1 -- 172.21.15.174:0/3986434264 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8030960 con 0x7f77cc065c60 2023-11-02T15:33:43.123 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:43.123 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 1 -- 172.21.15.174:0/3986434264 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030d30 con 0x7f77cc065c60 2023-11-02T15:33:43.123 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:43.123 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:43.124 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d0a64640 10 monclient: dump: 2023-11-02T15:33:43.124 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:43.124 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:43.124 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.124 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.125 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:43.125 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.125 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:43.125 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:43.125 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:43.126 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.126 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d3cca640 5 monclient: authenticate success, global_id 4212 2023-11-02T15:33:43.126 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d3cca640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:43.126 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.108+0000 7f77d3cca640 4 set_mon_vals no callback set 2023-11-02T15:33:43.126 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 10 monclient: shutdown 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 -- 172.21.15.174:0/3986434264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc065c60 msgr2=0x7f77cc13c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 --2- 172.21.15.174:0/3986434264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc065c60 0x7f77cc13c840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0addc0 tx=0x7f77c802b940 comp rx=0 tx=0).stop 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 -- 172.21.15.174:0/3986434264 shutdown_connections 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 --2- 172.21.15.174:0/3986434264 >> v2:172.21.15.175:3300/0 conn(0x7f77cc13fca0 0x7f77cc149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.127 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 --2- 172.21.15.174:0/3986434264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc065c60 0x7f77cc13c840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.128 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.109+0000 7f77d3cca640 1 -- 172.21.15.174:0/3986434264 >> 172.21.15.174:0/3986434264 conn(0x7f77cc06a550 msgr2=0x7f77cc0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:43.128 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.110+0000 7f77d3cca640 1 -- 172.21.15.174:0/3986434264 shutdown_connections 2023-11-02T15:33:43.128 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.110+0000 7f77d3cca640 1 -- 172.21.15.174:0/3986434264 wait complete. 2023-11-02T15:33:43.128 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 10 monclient: build_initial_monmap 2023-11-02T15:33:43.128 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 10 monclient: monmap: 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:43.129 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.130 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:43.130 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:43.130 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:43.130 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 librados: starting msgr at 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 librados: starting objecter 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 Processor -- start 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 -- start start 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 librados: setting wanted keys 2023-11-02T15:33:43.131 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 1 librados: calling monclient init 2023-11-02T15:33:43.132 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.111+0000 7f77d3cca640 10 monclient: init 2023-11-02T15:33:43.132 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:43.132 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:43.132 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.134 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-b con 0x7f77cc13cd80 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:43.134 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc13fca0 0x7f77cc0aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.135 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.112+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-a con 0x7f77cc13fca0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:43.135 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77cc0aacb0 0x7f77cc1d4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.136 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): picked mon.noname-c con 0x7f77cc0aacb0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:43.136 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:43.136 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0aca30 con 0x7f77cc13fca0 2023-11-02T15:33:43.136 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:43.137 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1437b0 con 0x7f77cc13cd80 2023-11-02T15:33:43.137 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:43.137 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0aa780 con 0x7f77cc0aacb0 2023-11-02T15:33:43.137 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:43.138 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d3cca640 10 monclient(hunting): authenticate will time out at 1350.454981s 2023-11-02T15:33:43.138 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.138 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:38722/0 (socket says 172.21.15.174:38722) 2023-11-02T15:33:43.138 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 1 -- 172.21.15.174:0/2940667456 learned_addr learned my addr 172.21.15.174:0/2940667456 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:43.138 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 10 monclient(hunting): get_auth_request con 0x7f77cc13cd80 auth_method 0 2023-11-02T15:33:43.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:43.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:43.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1a66640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:43.139 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1265640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc13fca0 0x7f77cc0aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.140 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1265640 10 monclient(hunting): get_auth_request con 0x7f77cc13fca0 auth_method 0 2023-11-02T15:33:43.140 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1265640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:43.140 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1265640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:43.140 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.113+0000 7f77d1265640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:43.140 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:43.141 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:43.141 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:43.141 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1265640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:43.141 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1265640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:43.141 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d1265640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:43.142 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d2267640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77cc0aacb0 0x7f77cc1d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.142 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d2267640 10 monclient(hunting): get_auth_request con 0x7f77cc0aacb0 auth_method 0 2023-11-02T15:33:43.142 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d2267640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:43.142 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d2267640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:43.142 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.114+0000 7f77d2267640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.115+0000 7f77d2267640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.115+0000 7f77d2267640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.115+0000 7f77d2267640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 10 monclient(hunting): handle_auth_done global_id 4207 payload 824 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 -- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77cc0aacb0 msgr2=0x7f77cc1d4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.143 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77cc0aacb0 0x7f77cc1d4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.144 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 -- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc13fca0 msgr2=0x7f77cc0aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.144 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc13fca0 0x7f77cc0aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.144 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 10 monclient: _finish_hunting 0 2023-11-02T15:33:43.144 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 monclient: found mon.noname-b 2023-11-02T15:33:43.145 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:43.145 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:33:43.145 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.117+0000 7f77d1a66640 1 -- 172.21.15.174:0/2940667456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80094d0 con 0x7f77cc13cd80 2023-11-02T15:33:43.147 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.119+0000 7f77d1a66640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009930 tx=0x7f77c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.119+0000 7f77c27fc640 10 client.?.objecter ms_handle_connect 0x7f77cc13cd80 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.119+0000 7f77c27fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8039c90 con 0x7f77cc13cd80 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:43.148 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: dump: 2023-11-02T15:33:43.149 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:43.149 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:43.149 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.149 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.149 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:43.150 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.150 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:43.150 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:43.150 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:43.150 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.151 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: _finish_auth 0 2023-11-02T15:33:43.151 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: _check_auth_tickets 2023-11-02T15:33:43.151 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:43.151 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8039df0 con 0x7f77cc13cd80 2023-11-02T15:33:43.151 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8043640 con 0x7f77cc13cd80 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c27fc640 10 monclient: dump: 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:43.152 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:43.153 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.153 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:43.153 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:43.153 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:43.153 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:43.154 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:43.154 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:43.154 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:43.154 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.123+0000 7f77c3fff640 4 set_mon_vals no callback set 2023-11-02T15:33:43.154 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 5 monclient: authenticate success, global_id 4207 2023-11-02T15:33:43.155 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 10 monclient: _renew_subs 2023-11-02T15:33:43.155 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:43.155 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc0a8e00 con 0x7f77cc13cd80 2023-11-02T15:33:43.155 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 10 client.4207.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:43.156 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 10 monclient: _renew_subs 2023-11-02T15:33:43.156 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:43.156 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc0ac6a0 con 0x7f77cc13cd80 2023-11-02T15:33:43.156 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.124+0000 7f77d3cca640 1 librados: init done 2023-11-02T15:33:43.156 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.125+0000 7f77d3cca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:43.157 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.125+0000 7f77d3cca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:43.157 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.125+0000 7f77d3cca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:43.157 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.125+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc067920 con 0x7f77cc13cd80 2023-11-02T15:33:43.157 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.143+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f77c80437a0 con 0x7f77cc13cd80 2023-11-02T15:33:43.158 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.143+0000 7f77c27fc640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 0x7f77b006e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:43.159 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.149+0000 7f77d1265640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 0x7f77b006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:43.159 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.149+0000 7f77d1265640 10 monclient: get_auth_request con 0x7f77b006bdb0 auth_method 0 2023-11-02T15:33:43.159 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.149+0000 7f77c27fc640 10 client.4207.objecter ms_handle_connect 0x7f77b006bdb0 2023-11-02T15:33:43.159 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.149+0000 7f77d1265640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 0x7f77b006e270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77cc143a40 tx=0x7f77bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:43.465 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f77c803e070 con 0x7f77cc13cd80 2023-11-02T15:33:43.466 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 10 client.4207.objecter ms_dispatch 0x7f77cc000b90 osd_map(10..10 src has 1..10) v4 2023-11-02T15:33:43.466 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 3 client.4207.objecter handle_osd_map got epochs [10,10] > 0 2023-11-02T15:33:43.466 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 3 client.4207.objecter handle_osd_map decoding full epoch 10 2023-11-02T15:33:43.466 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 20 client.4207.objecter dump_active .. 0 homeless 2023-11-02T15:33:43.467 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f77c80777e0 con 0x7f77cc13cd80 2023-11-02T15:33:43.467 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:43.467 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.464+0000 7f77c27fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:43.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.717+0000 7f77d3cca640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:43.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.717+0000 7f77d3cca640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:43.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.717+0000 7f77d3cca640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:33:43.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.717+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77cc0b2250 con 0x7f77cc13cd80 2023-11-02T15:33:43.746 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.745+0000 7f77c27fc640 1 -- 172.21.15.174:0/2940667456 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f77c807ae00 con 0x7f77cc13cd80 2023-11-02T15:33:43.747 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.745+0000 7f77c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:43.747 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.745+0000 7f77c27fc640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:43.747 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:43.747 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:10.458261+0000","last_up_change":"0.000000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:43.749 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.746+0000 7f77d3cca640 10 librados: watch_flush enter 2023-11-02T15:33:43.749 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.746+0000 7f77d3cca640 10 librados: watch_flush exit 2023-11-02T15:33:43.749 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 20 client.4207.objecter shutdown clearing up homeless session... 2023-11-02T15:33:43.750 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 10 client.4207.objecter successfully canceled tick 2023-11-02T15:33:43.750 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 msgr2=0x7f77b006e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.751 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 0x7f77b006e270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77cc143a40 tx=0x7f77bc009040 comp rx=0 tx=0).stop 2023-11-02T15:33:43.751 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 10 monclient: shutdown 2023-11-02T15:33:43.751 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:43.751 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 msgr2=0x7f77cc137e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:43.752 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009930 tx=0x7f77c8004730 comp rx=0 tx=0).stop 2023-11-02T15:33:43.752 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 shutdown_connections 2023-11-02T15:33:43.752 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f77b006bdb0 0x7f77b006e270 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77cc143a40 tx=0x7f77bc009040 comp rx=0 tx=0).stop 2023-11-02T15:33:43.752 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77cc0aacb0 0x7f77cc1d4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.753 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77cc13fca0 0x7f77cc0aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.753 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 --2- 172.21.15.174:0/2940667456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77cc13cd80 0x7f77cc137e10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:43.753 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.747+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 >> 172.21.15.174:0/2940667456 conn(0x7f77cc06a550 msgr2=0x7f77cc06f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:43.754 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.749+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 shutdown_connections 2023-11-02T15:33:43.754 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.749+0000 7f77d3cca640 1 -- 172.21.15.174:0/2940667456 wait complete. 2023-11-02T15:33:43.754 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:43.749+0000 7f77d3cca640 1 librados: shutdown 2023-11-02T15:33:43.763 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-02T15:33:45.778 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:33:45.773+0000 5dc2340 -1 osd.7 0 log_to_monitors true 2023-11-02T15:33:46.263 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:33:46.258+0000 5dc2340 -1 osd.3 0 log_to_monitors true 2023-11-02T15:33:46.477 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:33:46.468+0000 5dc2340 -1 osd.6 0 log_to_monitors true 2023-11-02T15:33:46.554 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:33:46.549+0000 5dc2340 -1 osd.4 0 log_to_monitors true 2023-11-02T15:33:46.913 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:33:46.908+0000 5dc2340 -1 osd.5 0 log_to_monitors true 2023-11-02T15:33:47.710 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:33:47.705+0000 5dc2340 -1 osd.1 0 log_to_monitors true 2023-11-02T15:33:48.063 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:33:48.049+0000 5dc2340 -1 osd.0 0 log_to_monitors true 2023-11-02T15:33:48.136 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:33:48.132+0000 5dc2340 -1 osd.2 0 log_to_monitors true 2023-11-02T15:33:48.806 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:33:48.799+0000 ac7d640 -1 osd.7 0 waiting for initial osdmap 2023-11-02T15:33:49.184 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:33:49.169+0000 14265640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:49.871 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:33:49.866+0000 c496640 -1 osd.5 0 waiting for initial osdmap 2023-11-02T15:33:49.877 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:33:49.872+0000 ac7d640 -1 osd.3 0 waiting for initial osdmap 2023-11-02T15:33:49.935 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:33:49.921+0000 ac7d640 -1 osd.6 0 waiting for initial osdmap 2023-11-02T15:33:49.951 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:33:49.944+0000 ac7d640 -1 osd.4 0 waiting for initial osdmap 2023-11-02T15:33:50.242 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:33:50.228+0000 14265640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:50.332 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:33:50.315+0000 14265640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:50.370 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:33:50.355+0000 14265640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:50.370 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:33:50.356+0000 14265640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:50.572 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:50.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.660+0000 7fe65ab1b640 10 monclient: get_monmap_and_config 2023-11-02T15:33:50.662 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.660+0000 7fe65ab1b640 10 monclient: build_initial_monmap 2023-11-02T15:33:50.663 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.660+0000 7fe65ab1b640 10 monclient: monmap: 2023-11-02T15:33:50.663 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:50.663 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:50.663 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:50.661653+0000 2023-11-02T15:33:50.664 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:50.661653+0000 2023-11-02T15:33:50.664 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:50.664 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.664 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:50.664 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:50.665 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:50.665 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:50.665 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:50.665 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.679 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.661+0000 7fe65ab1b640 1 Processor -- start 2023-11-02T15:33:50.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.661+0000 7fe65ab1b640 1 -- start start 2023-11-02T15:33:50.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.661+0000 7fe65ab1b640 10 monclient: init 2023-11-02T15:33:50.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:50.680 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2023-11-02T15:33:50.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe6540ae3a0 0x7fe65413e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:50.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-c con 0x7fe6540ae3a0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:50.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe6540b3600 0x7fe65413ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:50.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-a con 0x7fe6540b3600 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:50.681 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6540ad710 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:50.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6540add50 con 0x7fe6540ad710 2023-11-02T15:33:50.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:50.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6540ae780 con 0x7fe6540b3600 2023-11-02T15:33:50.682 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:50.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6540aef90 con 0x7fe6540ae3a0 2023-11-02T15:33:50.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:50.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe65ab1b640 10 monclient(hunting): authenticate will time out at 1358.003754s 2023-11-02T15:33:50.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.662+0000 7fe6590b8640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe6540ad710 0x7fe6541418c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:48734/0 (socket says 172.21.15.174:48734) 2023-11-02T15:33:50.683 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6590b8640 1 -- 172.21.15.174:0/876186102 learned_addr learned my addr 172.21.15.174:0/876186102 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:33:50.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6588b7640 1 --2- 172.21.15.174:0/876186102 >> v2:172.21.15.175:3300/0 conn(0x7fe6540ae3a0 0x7fe65413e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:50.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6588b7640 10 monclient(hunting): get_auth_request con 0x7fe6540ae3a0 auth_method 0 2023-11-02T15:33:50.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6588b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:50.684 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6588b7640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:50.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.663+0000 7fe6588b7640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:50.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.664+0000 7fe6588b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:50.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.664+0000 7fe6588b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:50.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.664+0000 7fe6588b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:50.685 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 10 monclient(hunting): handle_auth_done global_id 4234 payload 274 2023-11-02T15:33:50.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 1 -- 172.21.15.174:0/876186102 >> v2:172.21.15.174:3300/0 conn(0x7fe6540b3600 msgr2=0x7fe65413ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:50.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 1 --2- 172.21.15.174:0/876186102 >> v2:172.21.15.174:3300/0 conn(0x7fe6540b3600 0x7fe65413ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 1 -- 172.21.15.174:0/876186102 >> v1:172.21.15.175:6789/0 conn(0x7fe6540ad710 legacy=0x7fe6541418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:50.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 10 monclient: _finish_hunting 0 2023-11-02T15:33:50.686 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 1 monclient: found mon.noname-c 2023-11-02T15:33:50.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:50.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:33:50.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.666+0000 7fe6588b7640 1 -- 172.21.15.174:0/876186102 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540af100 con 0x7fe6540ae3a0 2023-11-02T15:33:50.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.669+0000 7fe6588b7640 1 --2- 172.21.15.174:0/876186102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ae3a0 0x7fe65413e930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6540b0060 tx=0x7fe64802b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a57c7038837b4c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:50.687 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 1 -- 172.21.15.174:0/876186102 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648031710 con 0x7fe6540ae3a0 2023-11-02T15:33:50.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:50.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:50.688 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: dump: 2023-11-02T15:33:50.688 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:50.688 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:50.689 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.689 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.689 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:50.689 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.689 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:50.690 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:50.690 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:50.690 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.690 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: _finish_auth 0 2023-11-02T15:33:50.690 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: _check_auth_tickets 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 5 monclient: authenticate success, global_id 4234 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 1 -- 172.21.15.174:0/876186102 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64802f050 con 0x7fe6540ae3a0 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:50.691 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:50.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 4 set_mon_vals no callback set 2023-11-02T15:33:50.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 1 -- 172.21.15.174:0/876186102 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648031a90 con 0x7fe6540ae3a0 2023-11-02T15:33:50.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:50.692 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:33:50.693 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe6537fe640 10 monclient: dump: 2023-11-02T15:33:50.693 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:50.693 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:50.693 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.693 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.694 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:50.694 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.694 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:50.694 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:50.695 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:50.695 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.695 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 10 monclient: shutdown 2023-11-02T15:33:50.695 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:50.695 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 -- 172.21.15.174:0/876186102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ae3a0 msgr2=0x7fe65413e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:50.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/876186102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ae3a0 0x7fe65413e930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6540b0060 tx=0x7fe64802b7b0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 -- 172.21.15.174:0/876186102 shutdown_connections 2023-11-02T15:33:50.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/876186102 >> v2:172.21.15.174:3300/0 conn(0x7fe6540b3600 0x7fe65413ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/876186102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ae3a0 0x7fe65413e930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.696 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 -- 172.21.15.174:0/876186102 >> 172.21.15.174:0/876186102 conn(0x7fe6540b7060 msgr2=0x7fe6540b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:50.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 -- 172.21.15.174:0/876186102 shutdown_connections 2023-11-02T15:33:50.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.673+0000 7fe65ab1b640 1 -- 172.21.15.174:0/876186102 wait complete. 2023-11-02T15:33:50.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 10 monclient: build_initial_monmap 2023-11-02T15:33:50.697 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 10 monclient: monmap: 2023-11-02T15:33:50.697 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:50.698 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:50.698 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:50.698 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:50.698 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:50.699 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.699 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:50.700 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:50.700 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:50.700 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.700 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 librados: starting msgr at 2023-11-02T15:33:50.700 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 librados: starting objecter 2023-11-02T15:33:50.701 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 Processor -- start 2023-11-02T15:33:50.701 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 -- start start 2023-11-02T15:33:50.701 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 librados: setting wanted keys 2023-11-02T15:33:50.701 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 1 librados: calling monclient init 2023-11-02T15:33:50.701 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.674+0000 7fe65ab1b640 10 monclient: init 2023-11-02T15:33:50.702 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:50.702 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:33:50.702 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ad710 0x7fe65413a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:50.702 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-b con 0x7fe6540ad710 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:50.703 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 0x7fe6541d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:50.703 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-c con 0x7fe6540ae3a0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:50.703 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 0x7fe6541371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:50.703 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): picked mon.noname-a con 0x7fe6540b3600 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:50.703 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:50.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe654145a60 con 0x7fe6540b3600 2023-11-02T15:33:50.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:50.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe654144970 con 0x7fe6540ad710 2023-11-02T15:33:50.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:50.704 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65413b240 con 0x7fe6540ae3a0 2023-11-02T15:33:50.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:50.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.675+0000 7fe65ab1b640 10 monclient(hunting): authenticate will time out at 1358.017197s 2023-11-02T15:33:50.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.676+0000 7fe653fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 0x7fe6541d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:50.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.677+0000 7fe6590b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 0x7fe6541371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:50.705 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 0x7fe6541d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:36744/0 (socket says 172.21.15.174:36744) 2023-11-02T15:33:50.706 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 1 -- 172.21.15.174:0/3516943168 learned_addr learned my addr 172.21.15.174:0/3516943168 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:50.706 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 10 monclient(hunting): get_auth_request con 0x7fe6540ae3a0 auth_method 0 2023-11-02T15:33:50.706 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:50.706 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:50.706 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.678+0000 7fe653fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:50.709 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.679+0000 7fe6590b8640 10 monclient(hunting): get_auth_request con 0x7fe6540b3600 auth_method 0 2023-11-02T15:33:50.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.679+0000 7fe6590b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:50.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.679+0000 7fe6590b8640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:50.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.679+0000 7fe6590b8640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:50.710 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe653fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:50.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe653fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:50.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe653fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:50.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe6590b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:50.711 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe6590b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:50.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.680+0000 7fe6590b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:50.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.682+0000 7fe6588b7640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ad710 0x7fe65413a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:50.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6588b7640 10 monclient(hunting): get_auth_request con 0x7fe6540ad710 auth_method 0 2023-11-02T15:33:50.714 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6588b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:50.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6588b7640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:50.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6588b7640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:50.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 10 monclient(hunting): handle_auth_done global_id 4236 payload 824 2023-11-02T15:33:50.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 -- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 msgr2=0x7fe6541d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:50.715 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 0x7fe6541d96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.716 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 -- 172.21.15.174:0/3516943168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ad710 msgr2=0x7fe65413a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:50.717 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ad710 0x7fe65413a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:50.717 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 10 monclient: _finish_hunting 0 2023-11-02T15:33:50.717 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 monclient: found mon.noname-a 2023-11-02T15:33:50.717 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:50.718 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:50.718 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.683+0000 7fe6590b8640 1 -- 172.21.15.174:0/3516943168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe648009400 con 0x7fe6540b3600 2023-11-02T15:33:50.718 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.685+0000 7fe6590b8640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 0x7fe6541371e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00caa0 tx=0x7fe64c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:50.718 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.685+0000 7fe6517fa640 10 client.?.objecter ms_handle_connect 0x7fe6540b3600 2023-11-02T15:33:50.718 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.686+0000 7fe6517fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:50.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.691+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00ac00 con 0x7fe6540b3600 2023-11-02T15:33:50.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.691+0000 7fe6517fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:50.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.691+0000 7fe6517fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:50.719 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.691+0000 7fe6517fa640 10 monclient: dump: 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:50.720 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.721 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:50.721 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:50.721 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:50.722 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.722 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: _finish_auth 0 2023-11-02T15:33:50.722 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: _check_auth_tickets 2023-11-02T15:33:50.722 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:50.722 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00ad60 con 0x7fe6540b3600 2023-11-02T15:33:50.723 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:50.723 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c012600 con 0x7fe6540b3600 2023-11-02T15:33:50.723 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 5 monclient: authenticate success, global_id 4236 2023-11-02T15:33:50.723 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 10 monclient: _renew_subs 2023-11-02T15:33:50.723 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:50.724 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe654143c10 con 0x7fe6540b3600 2023-11-02T15:33:50.724 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe652ffd640 4 set_mon_vals no callback set 2023-11-02T15:33:50.724 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:50.724 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:50.725 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe6517fa640 10 monclient: dump: 2023-11-02T15:33:50.725 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:50.725 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:50.725 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.726 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:50.726 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:50.726 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:50.727 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:50.727 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:50.727 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:50.728 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:50.728 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 10 client.4236.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:50.728 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 10 monclient: _renew_subs 2023-11-02T15:33:50.729 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:50.729 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe654144650 con 0x7fe6540b3600 2023-11-02T15:33:50.729 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.692+0000 7fe65ab1b640 1 librados: init done 2023-11-02T15:33:50.729 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.693+0000 7fe65ab1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:50.729 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.693+0000 7fe65ab1b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:50.730 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.693+0000 7fe65ab1b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:50.730 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:50.693+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65404e040 con 0x7fe6540b3600 2023-11-02T15:33:51.051 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.048+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7fe64c007070 con 0x7fe6540b3600 2023-11-02T15:33:51.051 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.049+0000 7fe6517fa640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 0x7fe63806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:51.052 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.049+0000 7fe6588b7640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 0x7fe63806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:51.052 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.049+0000 7fe6588b7640 10 monclient: get_auth_request con 0x7fe63806bd80 auth_method 0 2023-11-02T15:33:51.052 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.049+0000 7fe6517fa640 10 client.4236.objecter ms_handle_connect 0x7fe63806bd80 2023-11-02T15:33:51.052 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.049+0000 7fe6588b7640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 0x7fe63806e240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe648035000 tx=0x7fe648008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:51.090 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.089+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2463+0+0 (secure 0 0 0) 0x7fe64c057fc0 con 0x7fe6540b3600 2023-11-02T15:33:51.091 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.089+0000 7fe6517fa640 10 client.4236.objecter ms_dispatch 0x7fe654000b90 osd_map(13..13 src has 1..13) v4 2023-11-02T15:33:51.091 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.089+0000 7fe6517fa640 3 client.4236.objecter handle_osd_map got epochs [13,13] > 0 2023-11-02T15:33:51.091 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.089+0000 7fe6517fa640 3 client.4236.objecter handle_osd_map decoding full epoch 13 2023-11-02T15:33:51.092 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.089+0000 7fe6517fa640 20 client.4236.objecter dump_active .. 0 homeless 2023-11-02T15:33:51.092 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.091+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe64c054300 con 0x7fe6540b3600 2023-11-02T15:33:51.093 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.091+0000 7fe6517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:51.093 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.091+0000 7fe6517fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:51.297 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.295+0000 7fe65ab1b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:51.297 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.295+0000 7fe65ab1b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:51.297 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.295+0000 7fe65ab1b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:51.298 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.295+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6540b1f00 con 0x7fe6540b3600 2023-11-02T15:33:51.337 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.336+0000 7fe6517fa640 1 -- 172.21.15.174:0/3516943168 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9233 (secure 0 0 0) 0x7fe64c021090 con 0x7fe6540b3600 2023-11-02T15:33:51.338 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.336+0000 7fe6517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:51.338 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.336+0000 7fe6517fa640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:51.338 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:51.339 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":14,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:50.659000+0000","last_up_change":"2023-11-02T15:33:50.659000+0000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":86148},{"type":"v1","addr":"172.21.15.174:6820","nonce":86148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":86148},{"type":"v1","addr":"172.21.15.174:6823","nonce":86148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":86148},{"type":"v1","addr":"172.21.15.174:6829","nonce":86148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":86148},{"type":"v1","addr":"172.21.15.174:6826","nonce":86148}]},"public_addr":"172.21.15.174:6820/86148","cluster_addr":"172.21.15.174:6823/86148","heartbeat_back_addr":"172.21.15.174:6829/86148","heartbeat_front_addr":"172.21.15.174:6826/86148","state":["exists","up"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":86415},{"type":"v1","addr":"172.21.15.175:6817","nonce":86415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":86415},{"type":"v1","addr":"172.21.15.175:6819","nonce":86415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":86415},{"type":"v1","addr":"172.21.15.175:6823","nonce":86415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":86415},{"type":"v1","addr":"172.21.15.175:6821","nonce":86415}]},"public_addr":"172.21.15.175:6817/86415","cluster_addr":"172.21.15.175:6819/86415","heartbeat_back_addr":"172.21.15.175:6823/86415","heartbeat_front_addr":"172.21.15.175:6821/86415","state":["exists","up"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":86416},{"type":"v1","addr":"172.21.15.175:6809","nonce":86416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":86416},{"type":"v1","addr":"172.21.15.175:6811","nonce":86416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":86416},{"type":"v1","addr":"172.21.15.175:6815","nonce":86416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":86416},{"type":"v1","addr":"172.21.15.175:6813","nonce":86416}]},"public_addr":"172.21.15.175:6809/86416","cluster_addr":"172.21.15.175:6811/86416","heartbeat_back_addr":"172.21.15.175:6815/86416","heartbeat_front_addr":"172.21.15.175:6813/86416","state":["exists","up"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":86417},{"type":"v1","addr":"172.21.15.175:6825","nonce":86417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":86417},{"type":"v1","addr":"172.21.15.175:6827","nonce":86417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":86417},{"type":"v1","addr":"172.21.15.175:6831","nonce":86417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":86417},{"type":"v1","addr":"172.21.15.175:6829","nonce":86417}]},"public_addr":"172.21.15.175:6825/86417","cluster_addr":"172.21.15.175:6827/86417","heartbeat_back_addr":"172.21.15.175:6831/86417","heartbeat_front_addr":"172.21.15.175:6829/86417","state":["exists","up"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":86418},{"type":"v1","addr":"172.21.15.175:6801","nonce":86418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":86418},{"type":"v1","addr":"172.21.15.175:6803","nonce":86418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":86418},{"type":"v1","addr":"172.21.15.175:6807","nonce":86418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":86418},{"type":"v1","addr":"172.21.15.175:6805","nonce":86418}]},"public_addr":"172.21.15.175:6801/86418","cluster_addr":"172.21.15.175:6803/86418","heartbeat_back_addr":"172.21.15.175:6807/86418","heartbeat_front_addr":"172.21.15.175:6805/86418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:47.661882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:51.340 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.338+0000 7fe65ab1b640 10 librados: watch_flush enter 2023-11-02T15:33:51.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.338+0000 7fe65ab1b640 10 librados: watch_flush exit 2023-11-02T15:33:51.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 20 client.4236.objecter shutdown clearing up homeless session... 2023-11-02T15:33:51.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 10 client.4236.objecter successfully canceled tick 2023-11-02T15:33:51.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 msgr2=0x7fe63806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:51.341 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 0x7fe63806e240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe648035000 tx=0x7fe648008030 comp rx=0 tx=0).stop 2023-11-02T15:33:51.342 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 10 monclient: shutdown 2023-11-02T15:33:51.342 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:51.342 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 msgr2=0x7fe6541371e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:51.342 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 0x7fe6541371e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00caa0 tx=0x7fe64c00cf60 comp rx=0 tx=0).stop 2023-11-02T15:33:51.343 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 shutdown_connections 2023-11-02T15:33:51.343 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6540b3600 0x7fe6541371e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:51.343 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe63806bd80 0x7fe63806e240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:51.343 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6540ae3a0 0x7fe6541d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:51.343 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 --2- 172.21.15.174:0/3516943168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6540ad710 0x7fe65413a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:51.344 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 >> 172.21.15.174:0/3516943168 conn(0x7fe6540b7060 msgr2=0x7fe65413e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:51.344 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 shutdown_connections 2023-11-02T15:33:51.344 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 -- 172.21.15.174:0/3516943168 wait complete. 2023-11-02T15:33:51.344 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:51.339+0000 7fe65ab1b640 1 librados: shutdown 2023-11-02T15:33:51.354 DEBUG:teuthology.misc:5 of 8 OSDs are up 2023-11-02T15:33:51.465 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:33:51.460+0000 c496640 -1 osd.1 0 waiting for initial osdmap 2023-11-02T15:33:51.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:33:51.529+0000 ac7d640 -1 osd.2 0 waiting for initial osdmap 2023-11-02T15:33:51.559 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:33:51.554+0000 ac7d640 -1 osd.0 0 waiting for initial osdmap 2023-11-02T15:33:52.018 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:33:52.003+0000 14265640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:52.054 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:33:52.040+0000 14265640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:52.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:33:52.045+0000 14265640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:33:58.160 DEBUG:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-02T15:33:58.258 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 10 monclient: get_monmap_and_config 2023-11-02T15:33:58.259 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 10 monclient: build_initial_monmap 2023-11-02T15:33:58.259 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 10 monclient: monmap: 2023-11-02T15:33:58.259 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:58.260 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:58.260 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:33:58.258005+0000 2023-11-02T15:33:58.260 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:33:58.258005+0000 2023-11-02T15:33:58.260 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:58.260 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.261 INFO:teuthology.misc.health.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:58.261 INFO:teuthology.misc.health.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:58.261 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:58.261 INFO:teuthology.misc.health.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:58.261 INFO:teuthology.misc.health.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:58.262 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.262 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 1 Processor -- start 2023-11-02T15:33:58.262 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 1 -- start start 2023-11-02T15:33:58.262 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.256+0000 7f40745d3640 10 monclient: init 2023-11-02T15:33:58.268 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:58.268 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-11-02T15:33:58.268 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.268 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-b con 0x7f406c06f950 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:58.269 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f406c06fd30 addr v1:172.21.15.175:6789/0 2023-11-02T15:33:58.269 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 0x7f406c13fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.269 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-a con 0x7f406c13c9a0 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:58.270 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c070d00 con 0x7f406c06fd30 2023-11-02T15:33:58.270 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:58.271 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c13ce10 con 0x7f406c13c9a0 2023-11-02T15:33:58.271 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:58.271 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0718c0 con 0x7f406c06f950 2023-11-02T15:33:58.271 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:58.272 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f40745d3640 10 monclient(hunting): authenticate will time out at 1365.600198s 2023-11-02T15:33:58.272 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.257+0000 7f407236f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:58.272 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 0x7f406c13fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:58.272 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38752/0 (socket says 172.21.15.174:38752) 2023-11-02T15:33:58.272 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 1 -- 172.21.15.174:0/2776522923 learned_addr learned my addr 172.21.15.174:0/2776522923 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:58.273 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): get_auth_request con 0x7f406c06f950 auth_method 0 2023-11-02T15:33:58.273 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:58.273 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:58.273 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:58.273 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): get_auth_request con 0x7f406c13c9a0 auth_method 0 2023-11-02T15:33:58.274 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:58.274 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:58.274 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:58.274 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:58.274 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:58.275 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:58.275 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:58.275 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:58.276 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.258+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:58.276 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 10 monclient(hunting): handle_auth_done global_id 4226 payload 274 2023-11-02T15:33:58.277 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 1 -- 172.21.15.174:0/2776522923 >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 msgr2=0x7f406c13fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:58.277 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 1 --2- 172.21.15.174:0/2776522923 >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 0x7f406c13fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:58.277 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 1 -- 172.21.15.174:0/2776522923 >> v1:172.21.15.175:6789/0 conn(0x7f406c06fd30 legacy=0x7f406c13f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:58.277 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 10 monclient: _finish_hunting 0 2023-11-02T15:33:58.277 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 1 monclient: found mon.noname-b 2023-11-02T15:33:58.278 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:58.278 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.278 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f407236f640 1 -- 172.21.15.174:0/2776522923 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c071ba0 con 0x7f406c06f950 2023-11-02T15:33:58.278 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.260+0000 7f4072b70640 1 --2- 172.21.15.174:0/2776522923 >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 0x7f406c13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T15:33:58.279 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.262+0000 7f407236f640 1 --2- 172.21.15.174:0/2776522923 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f406802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c93241dae0540a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:58.279 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 1 -- 172.21.15.174:0/2776522923 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680317c0 con 0x7f406c06f950 2023-11-02T15:33:58.279 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:58.279 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:33:58.279 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: dump: 2023-11-02T15:33:58.280 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:58.280 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:58.280 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.280 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.280 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:58.281 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.281 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:58.281 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:58.281 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:58.281 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.282 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: _finish_auth 0 2023-11-02T15:33:58.282 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: _check_auth_tickets 2023-11-02T15:33:58.282 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:58.282 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 1 -- 172.21.15.174:0/2776522923 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068031920 con 0x7f406c06f950 2023-11-02T15:33:58.283 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f40745d3640 5 monclient: authenticate success, global_id 4226 2023-11-02T15:33:58.283 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:58.283 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f40745d3640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:58.283 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 1 -- 172.21.15.174:0/2776522923 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068031cb0 con 0x7f406c06f950 2023-11-02T15:33:58.284 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:58.284 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f40745d3640 4 set_mon_vals no callback set 2023-11-02T15:33:58.284 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:58.284 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.265+0000 7f407136d640 10 monclient: dump: 2023-11-02T15:33:58.284 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:58.285 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:58.285 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.285 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.285 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:58.285 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.286 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:58.286 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:58.286 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:58.286 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.294 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 10 monclient: shutdown 2023-11-02T15:33:58.294 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:58.294 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 -- 172.21.15.174:0/2776522923 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 msgr2=0x7f406c13c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:58.295 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 --2- 172.21.15.174:0/2776522923 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f406802b860 comp rx=0 tx=0).stop 2023-11-02T15:33:58.295 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 -- 172.21.15.174:0/2776522923 shutdown_connections 2023-11-02T15:33:58.295 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 --2- 172.21.15.174:0/2776522923 >> v2:172.21.15.174:3300/0 conn(0x7f406c13c9a0 0x7f406c13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:58.295 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 --2- 172.21.15.174:0/2776522923 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c13c460 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:58.296 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 -- 172.21.15.174:0/2776522923 >> 172.21.15.174:0/2776522923 conn(0x7f406c0b5000 msgr2=0x7f406c0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:58.296 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 -- 172.21.15.174:0/2776522923 shutdown_connections 2023-11-02T15:33:58.296 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.266+0000 7f40745d3640 1 -- 172.21.15.174:0/2776522923 wait complete. 2023-11-02T15:33:58.296 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 10 monclient: build_initial_monmap 2023-11-02T15:33:58.297 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 10 monclient: monmap: 2023-11-02T15:33:58.297 INFO:teuthology.misc.health.smithi174.stderr:epoch 0 2023-11-02T15:33:58.297 INFO:teuthology.misc.health.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:58.298 INFO:teuthology.misc.health.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:58.298 INFO:teuthology.misc.health.smithi174.stderr:created 0.000000 2023-11-02T15:33:58.298 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:58.298 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.298 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:58.299 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:58.299 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:58.299 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.300 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 librados: starting msgr at 2023-11-02T15:33:58.300 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 librados: starting objecter 2023-11-02T15:33:58.300 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 Processor -- start 2023-11-02T15:33:58.300 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 -- start start 2023-11-02T15:33:58.301 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 librados: setting wanted keys 2023-11-02T15:33:58.301 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 1 librados: calling monclient init 2023-11-02T15:33:58.301 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.267+0000 7f40745d3640 10 monclient: init 2023-11-02T15:33:58.301 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:58.302 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:33:58.302 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.303 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-c con 0x7f406c06f950 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:58.303 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406c06fd30 0x7f406c139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.303 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-b con 0x7f406c06fd30 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:58.303 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f406c13c9a0 0x7f406c139540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.304 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): picked mon.noname-a con 0x7f406c13c9a0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:58.304 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:58.304 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c141c20 con 0x7f406c13c9a0 2023-11-02T15:33:58.304 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:58.304 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c142bf0 con 0x7f406c06fd30 2023-11-02T15:33:58.305 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:58.305 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c1429f0 con 0x7f406c06f950 2023-11-02T15:33:58.305 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:58.306 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f40745d3640 10 monclient(hunting): authenticate will time out at 1365.611088s 2023-11-02T15:33:58.306 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f407236f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:58.306 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.268+0000 7f4072b70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f406c13c9a0 0x7f406c139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:58.306 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38758/0 (socket says 172.21.15.174:38758) 2023-11-02T15:33:58.307 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 1 -- 172.21.15.174:0/1606417777 learned_addr learned my addr 172.21.15.174:0/1606417777 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:58.307 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): get_auth_request con 0x7f406c06f950 auth_method 0 2023-11-02T15:33:58.307 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:58.308 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:58.308 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:58.308 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): get_auth_request con 0x7f406c13c9a0 auth_method 0 2023-11-02T15:33:58.309 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:58.309 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:58.309 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:58.309 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:58.309 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:58.310 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f407236f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:58.310 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:58.310 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:58.310 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.269+0000 7f4072b70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:58.310 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 10 monclient(hunting): handle_auth_done global_id 4229 payload 824 2023-11-02T15:33:58.311 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 -- 172.21.15.174:0/1606417777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406c06fd30 msgr2=0x7f406c139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:58.311 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406c06fd30 0x7f406c139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:58.311 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 -- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f406c13c9a0 msgr2=0x7f406c139540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:58.311 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f406c13c9a0 0x7f406c139540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:58.312 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 10 monclient: _finish_hunting 0 2023-11-02T15:33:58.312 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 monclient: found mon.noname-c 2023-11-02T15:33:58.312 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:58.313 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.313 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.273+0000 7f407236f640 1 -- 172.21.15.174:0/1606417777 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064009a70 con 0x7f406c06f950 2023-11-02T15:33:58.314 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.276+0000 7f407236f640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4068000c00 tx=0x7f40680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:58.314 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.276+0000 7f4062ffd640 10 client.?.objecter ms_handle_connect 0x7f406c06f950 2023-11-02T15:33:58.315 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.276+0000 7f4062ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:58.315 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068032860 con 0x7f406c06f950 2023-11-02T15:33:58.315 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:58.315 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:33:58.316 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: dump: 2023-11-02T15:33:58.316 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:58.316 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:58.316 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.317 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.317 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:58.317 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.317 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:58.317 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:58.318 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:58.318 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.318 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:58.318 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:58.318 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:58.319 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40680329e0 con 0x7f406c06f950 2023-11-02T15:33:58.319 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 5 monclient: authenticate success, global_id 4229 2023-11-02T15:33:58.319 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 10 monclient: _renew_subs 2023-11-02T15:33:58.319 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.319 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40680094d0 con 0x7f406c06f950 2023-11-02T15:33:58.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:58.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068032d90 con 0x7f406c06f950 2023-11-02T15:33:58.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:58.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:33:58.320 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 10 client.4229.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:58.321 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4062ffd640 10 monclient: dump: 2023-11-02T15:33:58.321 INFO:teuthology.misc.health.smithi174.stderr:epoch 1 2023-11-02T15:33:58.321 INFO:teuthology.misc.health.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:58.321 INFO:teuthology.misc.health.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.322 INFO:teuthology.misc.health.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:58.322 INFO:teuthology.misc.health.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:58.322 INFO:teuthology.misc.health.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:58.322 INFO:teuthology.misc.health.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:58.322 INFO:teuthology.misc.health.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:58.323 INFO:teuthology.misc.health.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:58.323 INFO:teuthology.misc.health.smithi174.stderr: 2023-11-02T15:33:58.323 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 10 monclient: _renew_subs 2023-11-02T15:33:58.323 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.323 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f4070b6c640 4 set_mon_vals no callback set 2023-11-02T15:33:58.324 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c1435b0 con 0x7f406c06f950 2023-11-02T15:33:58.324 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.280+0000 7f40745d3640 1 librados: init done 2023-11-02T15:33:58.324 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.281+0000 7f40745d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:58.324 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.281+0000 7f40745d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:58.324 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.281+0000 7f40745d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.325 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.281+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c142710 con 0x7f406c06f950 2023-11-02T15:33:58.326 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.298+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f406804e450 con 0x7f406c06f950 2023-11-02T15:33:58.326 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.299+0000 7f4062ffd640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 0x7f405006e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:58.326 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.299+0000 7f4071b6e640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 0x7f405006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:58.328 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.299+0000 7f4071b6e640 10 monclient: get_auth_request con 0x7f405006bd80 auth_method 0 2023-11-02T15:33:58.329 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.299+0000 7f4062ffd640 10 client.4229.objecter ms_handle_connect 0x7f405006bd80 2023-11-02T15:33:58.329 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.299+0000 7f4071b6e640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 0x7f405006e240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f406c0a92e0 tx=0x7f405c007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:58.329 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.301+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f406807a650 con 0x7f406c06f950 2023-11-02T15:33:58.329 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.301+0000 7f4062ffd640 10 client.4229.objecter ms_dispatch 0x7f406c000b90 osd_map(19..19 src has 1..19) v4 2023-11-02T15:33:58.329 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.301+0000 7f4062ffd640 3 client.4229.objecter handle_osd_map got epochs [19,19] > 0 2023-11-02T15:33:58.330 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.301+0000 7f4062ffd640 3 client.4229.objecter handle_osd_map decoding full epoch 19 2023-11-02T15:33:58.330 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.302+0000 7f4062ffd640 20 client.4229.objecter dump_active .. 0 homeless 2023-11-02T15:33:58.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.579+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4068076290 con 0x7f406c06f950 2023-11-02T15:33:58.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.579+0000 7f4062ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:58.582 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.579+0000 7f4062ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:58.787 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.784+0000 7f40745d3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:58.787 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.784+0000 7f40745d3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:58.787 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.784+0000 7f40745d3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:33:58.787 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:58.784+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f406c067920 con 0x7f406c06f950 2023-11-02T15:33:59.033 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.030+0000 7f4062ffd640 1 -- 172.21.15.174:0/1606417777 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f40680798b0 con 0x7f406c06f950 2023-11-02T15:33:59.033 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.030+0000 7f4062ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:33:59.033 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.030+0000 7f4062ffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:33:59.034 INFO:teuthology.misc.health.smithi174.stdout: 2023-11-02T15:33:59.034 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":19,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:33:56.418411+0000","last_up_change":"2023-11-02T15:33:53.161912+0000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-02T15:33:53.130231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":86149},{"type":"v1","addr":"172.21.15.174:6803","nonce":86149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":86149},{"type":"v1","addr":"172.21.15.174:6805","nonce":86149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":86149},{"type":"v1","addr":"172.21.15.174:6809","nonce":86149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":86149},{"type":"v1","addr":"172.21.15.174:6807","nonce":86149}]},"public_addr":"172.21.15.174:6803/86149","cluster_addr":"172.21.15.174:6805/86149","heartbeat_back_addr":"172.21.15.174:6809/86149","heartbeat_front_addr":"172.21.15.174:6807/86149","state":["exists","up"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":86146},{"type":"v1","addr":"172.21.15.174:6821","nonce":86146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":86146},{"type":"v1","addr":"172.21.15.174:6827","nonce":86146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":86146},{"type":"v1","addr":"172.21.15.174:6833","nonce":86146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":86146},{"type":"v1","addr":"172.21.15.174:6831","nonce":86146}]},"public_addr":"172.21.15.174:6821/86146","cluster_addr":"172.21.15.174:6827/86146","heartbeat_back_addr":"172.21.15.174:6833/86146","heartbeat_front_addr":"172.21.15.174:6831/86146","state":["exists","up"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":86153},{"type":"v1","addr":"172.21.15.174:6811","nonce":86153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":86153},{"type":"v1","addr":"172.21.15.174:6813","nonce":86153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":86153},{"type":"v1","addr":"172.21.15.174:6817","nonce":86153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":86153},{"type":"v1","addr":"172.21.15.174:6815","nonce":86153}]},"public_addr":"172.21.15.174:6811/86153","cluster_addr":"172.21.15.174:6813/86153","heartbeat_back_addr":"172.21.15.174:6817/86153","heartbeat_front_addr":"172.21.15.174:6815/86153","state":["exists","up"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":86148},{"type":"v1","addr":"172.21.15.174:6820","nonce":86148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":86148},{"type":"v1","addr":"172.21.15.174:6823","nonce":86148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":86148},{"type":"v1","addr":"172.21.15.174:6829","nonce":86148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":86148},{"type":"v1","addr":"172.21.15.174:6826","nonce":86148}]},"public_addr":"172.21.15.174:6820/86148","cluster_addr":"172.21.15.174:6823/86148","heartbeat_back_addr":"172.21.15.174:6829/86148","heartbeat_front_addr":"172.21.15.174:6826/86148","state":["exists","up"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":86415},{"type":"v1","addr":"172.21.15.175:6817","nonce":86415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":86415},{"type":"v1","addr":"172.21.15.175:6819","nonce":86415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":86415},{"type":"v1","addr":"172.21.15.175:6823","nonce":86415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":86415},{"type":"v1","addr":"172.21.15.175:6821","nonce":86415}]},"public_addr":"172.21.15.175:6817/86415","cluster_addr":"172.21.15.175:6819/86415","heartbeat_back_addr":"172.21.15.175:6823/86415","heartbeat_front_addr":"172.21.15.175:6821/86415","state":["exists","up"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":86416},{"type":"v1","addr":"172.21.15.175:6809","nonce":86416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":86416},{"type":"v1","addr":"172.21.15.175:6811","nonce":86416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":86416},{"type":"v1","addr":"172.21.15.175:6815","nonce":86416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":86416},{"type":"v1","addr":"172.21.15.175:6813","nonce":86416}]},"public_addr":"172.21.15.175:6809/86416","cluster_addr":"172.21.15.175:6811/86416","heartbeat_back_addr":"172.21.15.175:6815/86416","heartbeat_front_addr":"172.21.15.175:6813/86416","state":["exists","up"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":86417},{"type":"v1","addr":"172.21.15.175:6825","nonce":86417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":86417},{"type":"v1","addr":"172.21.15.175:6827","nonce":86417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":86417},{"type":"v1","addr":"172.21.15.175:6831","nonce":86417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":86417},{"type":"v1","addr":"172.21.15.175:6829","nonce":86417}]},"public_addr":"172.21.15.175:6825/86417","cluster_addr":"172.21.15.175:6827/86417","heartbeat_back_addr":"172.21.15.175:6831/86417","heartbeat_front_addr":"172.21.15.175:6829/86417","state":["exists","up"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":86418},{"type":"v1","addr":"172.21.15.175:6801","nonce":86418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":86418},{"type":"v1","addr":"172.21.15.175:6803","nonce":86418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":86418},{"type":"v1","addr":"172.21.15.175:6807","nonce":86418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":86418},{"type":"v1","addr":"172.21.15.175:6805","nonce":86418}]},"public_addr":"172.21.15.175:6801/86418","cluster_addr":"172.21.15.175:6803/86418","heartbeat_back_addr":"172.21.15.175:6807/86418","heartbeat_front_addr":"172.21.15.175:6805/86418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.105568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:49.744420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.135477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.453210+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.626045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.898023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.470083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:47.661882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:33:59.035 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 10 librados: watch_flush enter 2023-11-02T15:33:59.036 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 10 librados: watch_flush exit 2023-11-02T15:33:59.036 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 20 client.4229.objecter shutdown clearing up homeless session... 2023-11-02T15:33:59.036 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 10 client.4229.objecter successfully canceled tick 2023-11-02T15:33:59.036 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 msgr2=0x7f405006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:59.037 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 0x7f405006e240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f406c0a92e0 tx=0x7f405c007990 comp rx=0 tx=0).stop 2023-11-02T15:33:59.037 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 10 monclient: shutdown 2023-11-02T15:33:59.037 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:59.037 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 msgr2=0x7f406c06c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:59.038 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4068000c00 tx=0x7f40680046c0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.038 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 shutdown_connections 2023-11-02T15:33:59.038 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f406c13c9a0 0x7f406c139540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.038 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406c06fd30 0x7f406c139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.039 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.032+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f405006bd80 0x7f405006e240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.042 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.033+0000 7f40745d3640 1 --2- 172.21.15.174:0/1606417777 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f406c06f950 0x7f406c06c720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.042 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.033+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 >> 172.21.15.174:0/1606417777 conn(0x7f406c0b5000 msgr2=0x7f406c1492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:59.042 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.033+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 shutdown_connections 2023-11-02T15:33:59.043 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.033+0000 7f40745d3640 1 -- 172.21.15.174:0/1606417777 wait complete. 2023-11-02T15:33:59.043 INFO:teuthology.misc.health.smithi174.stderr:2023-11-02T15:33:59.033+0000 7f40745d3640 1 librados: shutdown 2023-11-02T15:33:59.045 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-02T15:33:59.046 INFO:tasks.ceph:Creating RBD pool 2023-11-02T15:33:59.046 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-02T15:33:59.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.146+0000 7ff7583dd640 10 monclient: get_monmap_and_config 2023-11-02T15:33:59.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.146+0000 7ff7583dd640 10 monclient: build_initial_monmap 2023-11-02T15:33:59.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.146+0000 7ff7583dd640 10 monclient: monmap: 2023-11-02T15:33:59.149 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:33:59.149 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:59.150 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:33:59.148381+0000 2023-11-02T15:33:59.150 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:33:59.148381+0000 2023-11-02T15:33:59.150 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:59.150 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.150 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:33:59.151 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:33:59.151 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:33:59.151 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:33:59.151 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:33:59.151 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 Processor -- start 2023-11-02T15:33:59.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 -- start start 2023-11-02T15:33:59.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient: init 2023-11-02T15:33:59.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:59.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2023-11-02T15:33:59.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff750144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-b con 0x7ff7500b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:59.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7500b0ed0 0x7ff7500a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-c con 0x7ff7500b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:33:59.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7500b1cb0 0x7ff7500b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-a con 0x7ff7500b1cb0 addr v2:172.21.15.174:3300/0 2023-11-02T15:33:59.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500b4950 con 0x7ff7500b1cb0 2023-11-02T15:33:59.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500aa1d0 con 0x7ff7500b0ed0 2023-11-02T15:33:59.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500aa4b0 con 0x7ff7500b4570 2023-11-02T15:33:59.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:59.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.147+0000 7ff7583dd640 10 monclient(hunting): authenticate will time out at 1366.490580s 2023-11-02T15:33:59.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.194+0000 7ff75697a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7500b1cb0 0x7ff7500b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:59.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7500b1cb0 0x7ff7500b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38706/0 (socket says 172.21.15.174:38706) 2023-11-02T15:33:59.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 1 -- 172.21.15.174:0/610377395 learned_addr learned my addr 172.21.15.174:0/610377395 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:59.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 10 monclient(hunting): get_auth_request con 0x7ff7500b1cb0 auth_method 0 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.195+0000 7ff75697a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.196+0000 7ff75697a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.196+0000 7ff75697a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:59.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.196+0000 7ff75697a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:59.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 10 monclient(hunting): handle_auth_done global_id 4251 payload 274 2023-11-02T15:33:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 -- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 msgr2=0x7ff750144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 --2- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff750144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 -- 172.21.15.174:0/610377395 >> v2:172.21.15.175:3300/0 conn(0x7ff7500b0ed0 msgr2=0x7ff7500a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:33:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 --2- 172.21.15.174:0/610377395 >> v2:172.21.15.175:3300/0 conn(0x7ff7500b0ed0 0x7ff7500a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 10 monclient: _finish_hunting 0 2023-11-02T15:33:59.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 monclient: found mon.noname-a 2023-11-02T15:33:59.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:59.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.197+0000 7ff75697a640 1 -- 172.21.15.174:0/610377395 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500a8b50 con 0x7ff7500b1cb0 2023-11-02T15:33:59.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.199+0000 7ff75697a640 1 --2- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff7500b2120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b8390 tx=0x7ff74802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f0eb9014322534 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:59.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 1 -- 172.21.15.174:0/610377395 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74802c950 con 0x7ff7500b1cb0 2023-11-02T15:33:59.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:59.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:59.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: dump: 2023-11-02T15:33:59.207 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:59.207 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:59.208 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.208 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.208 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:59.208 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: _finish_auth 0 2023-11-02T15:33:59.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: _check_auth_tickets 2023-11-02T15:33:59.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:59.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 1 -- 172.21.15.174:0/610377395 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74802cab0 con 0x7ff7500b1cb0 2023-11-02T15:33:59.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:59.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 1 -- 172.21.15.174:0/610377395 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74802ce40 con 0x7ff7500b1cb0 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff7583dd640 5 monclient: authenticate success, global_id 4251 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff7583dd640 10 monclient: get_monmap_and_config success 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff7583dd640 4 set_mon_vals no callback set 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.204+0000 7ff755177640 10 monclient: dump: 2023-11-02T15:33:59.212 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:59.212 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:59.212 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.212 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.212 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:59.213 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.213 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:59.213 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:59.213 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:59.213 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 10 monclient: shutdown 2023-11-02T15:33:59.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:33:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 -- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 msgr2=0x7ff7500b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 --2- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff7500b2120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b8390 tx=0x7ff74802bdb0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 -- 172.21.15.174:0/610377395 shutdown_connections 2023-11-02T15:33:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 --2- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff7500b2120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 --2- 172.21.15.174:0/610377395 >> v2:172.21.15.175:3300/0 conn(0x7ff7500b0ed0 0x7ff7500a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 --2- 172.21.15.174:0/610377395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff750144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 -- 172.21.15.174:0/610377395 >> 172.21.15.174:0/610377395 conn(0x7ff750065d50 msgr2=0x7ff750066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:33:59.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 -- 172.21.15.174:0/610377395 shutdown_connections 2023-11-02T15:33:59.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.205+0000 7ff7583dd640 1 -- 172.21.15.174:0/610377395 wait complete. 2023-11-02T15:33:59.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 10 monclient: build_initial_monmap 2023-11-02T15:33:59.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 10 monclient: monmap: 2023-11-02T15:33:59.222 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:33:59.222 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:33:59.222 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:33:59.223 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:33:59.223 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:33:59.223 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:33:59.224 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:33:59.224 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:33:59.224 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 librados: starting msgr at 2023-11-02T15:33:59.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 librados: starting objecter 2023-11-02T15:33:59.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 Processor -- start 2023-11-02T15:33:59.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 -- start start 2023-11-02T15:33:59.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 librados: setting wanted keys 2023-11-02T15:33:59.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 1 librados: calling monclient init 2023-11-02T15:33:59.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.206+0000 7ff7583dd640 10 monclient: init 2023-11-02T15:33:59.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:33:59.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:33:59.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500b0ed0 0x7ff7500b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-b con 0x7ff7500b0ed0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:33:59.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-a con 0x7ff7500b1cb0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:33:59.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff7501366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): picked mon.noname-c con 0x7ff7500b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:33:59.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750141b70 con 0x7ff7500b1cb0 2023-11-02T15:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750141d90 con 0x7ff7500b0ed0 2023-11-02T15:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff750140b70 con 0x7ff7500b4570 2023-11-02T15:33:59.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:33:59.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff7583dd640 10 monclient(hunting): authenticate will time out at 1366.549850s 2023-11-02T15:33:59.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:59.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38710/0 (socket says 172.21.15.174:38710) 2023-11-02T15:33:59.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 1 -- 172.21.15.174:0/878518741 learned_addr learned my addr 172.21.15.174:0/878518741 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 10 monclient(hunting): get_auth_request con 0x7ff7500b1cb0 auth_method 0 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.207+0000 7ff755978640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.208+0000 7ff755978640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.208+0000 7ff755978640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.208+0000 7ff755978640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 10 monclient(hunting): handle_auth_done global_id 4254 payload 824 2023-11-02T15:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 -- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 msgr2=0x7ff7501366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff7501366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 -- 172.21.15.174:0/878518741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500b0ed0 msgr2=0x7ff7500b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:33:59.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500b0ed0 0x7ff7500b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:33:59.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 10 monclient: _finish_hunting 0 2023-11-02T15:33:59.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 monclient: found mon.noname-a 2023-11-02T15:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.211+0000 7ff755978640 1 -- 172.21.15.174:0/878518741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748009b30 con 0x7ff7500b1cb0 2023-11-02T15:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.213+0000 7ff755978640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b4a0 tx=0x7ff74000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.213+0000 7ff746ffd640 10 client.?.objecter ms_handle_connect 0x7ff7500b1cb0 2023-11-02T15:33:59.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.213+0000 7ff746ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:33:59.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740005a10 con 0x7ff7500b1cb0 2023-11-02T15:33:59.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:59.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:33:59.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: dump: 2023-11-02T15:33:59.238 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:59.238 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:59.238 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.238 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.241 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:59.241 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.242 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:59.242 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:59.242 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:59.243 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: _finish_auth 0 2023-11-02T15:33:59.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:33:59.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:33:59.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740005b70 con 0x7ff7500b1cb0 2023-11-02T15:33:59.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:33:59.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740010610 con 0x7ff7500b1cb0 2023-11-02T15:33:59.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 5 monclient: authenticate success, global_id 4254 2023-11-02T15:33:59.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 10 monclient: _renew_subs 2023-11-02T15:33:59.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff750140df0 con 0x7ff7500b1cb0 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff754976640 4 set_mon_vals no callback set 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff746ffd640 10 monclient: dump: 2023-11-02T15:33:59.245 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:33:59.246 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:33:59.246 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.246 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:33:59.246 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:33:59.246 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:33:59.247 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:33:59.247 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:33:59.247 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:33:59.247 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:33:59.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 10 client.4254.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:33:59.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 10 monclient: _renew_subs 2023-11-02T15:33:59.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff750136df0 con 0x7ff7500b1cb0 2023-11-02T15:33:59.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.217+0000 7ff7583dd640 1 librados: init done 2023-11-02T15:33:59.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.219+0000 7ff7583dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:59.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.219+0000 7ff7583dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:59.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.219+0000 7ff7583dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.219+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750067920 con 0x7ff7500b1cb0 2023-11-02T15:33:59.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.247+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff740010770 con 0x7ff7500b1cb0 2023-11-02T15:33:59.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.248+0000 7ff746ffd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 0x7ff73406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:33:59.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.248+0000 7ff756179640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 0x7ff73406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:33:59.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.248+0000 7ff756179640 10 monclient: get_auth_request con 0x7ff73406be10 auth_method 0 2023-11-02T15:33:59.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.248+0000 7ff756179640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 0x7ff73406e2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009670 tx=0x7ff74c006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:33:59.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.248+0000 7ff746ffd640 10 client.4254.objecter ms_handle_connect 0x7ff73406be10 2023-11-02T15:33:59.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.252+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff740057fb0 con 0x7ff7500b1cb0 2023-11-02T15:33:59.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.252+0000 7ff746ffd640 10 client.4254.objecter ms_dispatch 0x7ff750000b90 osd_map(19..19 src has 1..19) v4 2023-11-02T15:33:59.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.252+0000 7ff746ffd640 3 client.4254.objecter handle_osd_map got epochs [19,19] > 0 2023-11-02T15:33:59.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.252+0000 7ff746ffd640 3 client.4254.objecter handle_osd_map decoding full epoch 19 2023-11-02T15:33:59.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.252+0000 7ff746ffd640 20 client.4254.objecter dump_active .. 0 homeless 2023-11-02T15:33:59.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.551+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff740057680 con 0x7ff7500b1cb0 2023-11-02T15:33:59.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.551+0000 7ff746ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:33:59.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.551+0000 7ff746ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:33:59.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.727+0000 7ff7583dd640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-11-02T15:33:59.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.727+0000 7ff7583dd640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-11-02T15:33:59.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.727+0000 7ff7583dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:33:59.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:33:59.727+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7ff750141f50 con 0x7ff7500b1cb0 2023-11-02T15:34:00.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.140+0000 7ff746ffd640 1 -- 172.21.15.174:0/878518741 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v20) v1 ==== 109+0+0 (secure 0 0 0) 0x7ff740014070 con 0x7ff7500b1cb0 2023-11-02T15:34:00.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.140+0000 7ff746ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2023-11-02T15:34:00.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.140+0000 7ff746ffd640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2023-11-02T15:34:00.144 INFO:teuthology.orchestra.run.smithi174.stderr:pool 'rbd' created 2023-11-02T15:34:00.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 10 librados: watch_flush enter 2023-11-02T15:34:00.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 10 librados: watch_flush exit 2023-11-02T15:34:00.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 20 client.4254.objecter shutdown clearing up homeless session... 2023-11-02T15:34:00.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 10 client.4254.objecter successfully canceled tick 2023-11-02T15:34:00.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 msgr2=0x7ff73406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 0x7ff73406e2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009670 tx=0x7ff74c006bf0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 10 monclient: shutdown 2023-11-02T15:34:00.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:00.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 msgr2=0x7ff750141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b4a0 tx=0x7ff74000b960 comp rx=0 tx=0).stop 2023-11-02T15:34:00.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 shutdown_connections 2023-11-02T15:34:00.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff73406be10 0x7ff73406e2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7500b4570 0x7ff7501366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7500b1cb0 0x7ff750141630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.150 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 --2- 172.21.15.174:0/878518741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500b0ed0 0x7ff7500b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.150 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 >> 172.21.15.174:0/878518741 conn(0x7ff750065d50 msgr2=0x7ff75013ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:00.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 shutdown_connections 2023-11-02T15:34:00.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.142+0000 7ff7583dd640 1 -- 172.21.15.174:0/878518741 wait complete. 2023-11-02T15:34:00.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.143+0000 7ff7583dd640 1 librados: shutdown 2023-11-02T15:34:00.157 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-02T15:34:00.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.258+0000 7f098b29d640 10 monclient: get_monmap_and_config 2023-11-02T15:34:00.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.258+0000 7f098b29d640 10 monclient: build_initial_monmap 2023-11-02T15:34:00.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.258+0000 7f098b29d640 10 monclient: monmap: 2023-11-02T15:34:00.262 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:00.262 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:00.263 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:00.260848+0000 2023-11-02T15:34:00.263 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:00.260848+0000 2023-11-02T15:34:00.263 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:00.263 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.263 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:00.264 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:00.264 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:00.264 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:00.264 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:00.264 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.259+0000 7f098b29d640 1 Processor -- start 2023-11-02T15:34:00.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.259+0000 7f098b29d640 1 -- start start 2023-11-02T15:34:00.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.259+0000 7f098b29d640 10 monclient: init 2023-11-02T15:34:00.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:00.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2023-11-02T15:34:00.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09840b3c10 0x7f098413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-c con 0x7f09840b3c10 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:00.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f098413ede0 0x7f09840b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-a con 0x7f098413ede0 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f09840b0a00 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840af970 con 0x7f09840b0a00 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840b3ff0 con 0x7f098413ede0 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:00.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098413f250 con 0x7f09840b3c10 2023-11-02T15:34:00.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:00.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f098b29d640 10 monclient(hunting): authenticate will time out at 1367.602995s 2023-11-02T15:34:00.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.260+0000 7f0988838640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f098413ede0 0x7f09840b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0989039640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09840b3c10 0x7f098413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f098413ede0 0x7f09840b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38714/0 (socket says 172.21.15.174:38714) 2023-11-02T15:34:00.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 1 -- 172.21.15.174:0/110316263 learned_addr learned my addr 172.21.15.174:0/110316263 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:00.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): get_auth_request con 0x7f098413ede0 auth_method 0 2023-11-02T15:34:00.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:00.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:00.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:00.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:00.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:00.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.261+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:00.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.262+0000 7f0989039640 10 monclient(hunting): get_auth_request con 0x7f09840b3c10 auth_method 0 2023-11-02T15:34:00.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.262+0000 7f0989039640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:00.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.262+0000 7f0989039640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:00.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.262+0000 7f0989039640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:00.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:00.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:00.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:00.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 10 monclient(hunting): handle_auth_done global_id 4257 payload 274 2023-11-02T15:34:00.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 1 -- 172.21.15.174:0/110316263 >> v2:172.21.15.175:3300/0 conn(0x7f09840b3c10 msgr2=0x7f098413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:00.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 1 --2- 172.21.15.174:0/110316263 >> v2:172.21.15.175:3300/0 conn(0x7f09840b3c10 0x7f098413e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 1 -- 172.21.15.174:0/110316263 >> v1:172.21.15.175:6789/0 conn(0x7f09840b0a00 legacy=0x7f09840b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 10 monclient: _finish_hunting 0 2023-11-02T15:34:00.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 1 monclient: found mon.noname-a 2023-11-02T15:34:00.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:00.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.263+0000 7f0988838640 1 -- 172.21.15.174:0/110316263 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840afbc0 con 0x7f098413ede0 2023-11-02T15:34:00.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.265+0000 7f0988838640 1 --2- 172.21.15.174:0/110316263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f098413ede0 0x7f09840b04c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09840b4ac0 tx=0x7f097402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7b2067c72f659d server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:00.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 1 -- 172.21.15.174:0/110316263 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974031870 con 0x7f098413ede0 2023-11-02T15:34:00.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:00.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:00.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: dump: 2023-11-02T15:34:00.289 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:00.289 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:00.289 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.290 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.290 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:00.290 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.291 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:00.291 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:00.291 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:00.291 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: _finish_auth 0 2023-11-02T15:34:00.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:00.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:00.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 1 -- 172.21.15.174:0/110316263 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740319d0 con 0x7f098413ede0 2023-11-02T15:34:00.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:00.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 1 -- 172.21.15.174:0/110316263 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974031d60 con 0x7f098413ede0 2023-11-02T15:34:00.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:00.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:00.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f097bfff640 10 monclient: dump: 2023-11-02T15:34:00.293 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:00.293 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:00.294 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.294 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.294 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:00.294 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.294 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:00.295 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:00.295 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:00.295 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f098b29d640 5 monclient: authenticate success, global_id 4257 2023-11-02T15:34:00.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f098b29d640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:00.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.269+0000 7f098b29d640 4 set_mon_vals no callback set 2023-11-02T15:34:00.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 10 monclient: shutdown 2023-11-02T15:34:00.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:00.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 -- 172.21.15.174:0/110316263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f098413ede0 msgr2=0x7f09840b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 --2- 172.21.15.174:0/110316263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f098413ede0 0x7f09840b04c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09840b4ac0 tx=0x7f097402b910 comp rx=0 tx=0).stop 2023-11-02T15:34:00.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 -- 172.21.15.174:0/110316263 shutdown_connections 2023-11-02T15:34:00.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 --2- 172.21.15.174:0/110316263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f098413ede0 0x7f09840b04c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 --2- 172.21.15.174:0/110316263 >> v2:172.21.15.175:3300/0 conn(0x7f09840b3c10 0x7f098413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 -- 172.21.15.174:0/110316263 >> 172.21.15.174:0/110316263 conn(0x7f0984065960 msgr2=0x7f0984065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:00.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 -- 172.21.15.174:0/110316263 shutdown_connections 2023-11-02T15:34:00.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.270+0000 7f098b29d640 1 -- 172.21.15.174:0/110316263 wait complete. 2023-11-02T15:34:00.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 10 monclient: build_initial_monmap 2023-11-02T15:34:00.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 10 monclient: monmap: 2023-11-02T15:34:00.298 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:00.299 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:00.299 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:00.299 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:00.299 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:00.300 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.300 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:00.301 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:00.301 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:00.301 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 librados: starting msgr at 2023-11-02T15:34:00.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 librados: starting objecter 2023-11-02T15:34:00.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 Processor -- start 2023-11-02T15:34:00.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 -- start start 2023-11-02T15:34:00.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 librados: setting wanted keys 2023-11-02T15:34:00.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 1 librados: calling monclient init 2023-11-02T15:34:00.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 10 monclient: init 2023-11-02T15:34:00.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:00.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.271+0000 7f098b29d640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:00.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09840b0a00 0x7f09840ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-b con 0x7f09840b0a00 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:00.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-a con 0x7f09840b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:00.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 0x7f09841d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): picked mon.noname-c con 0x7f098413ede0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:00.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841d6720 con 0x7f09840b3c10 2023-11-02T15:34:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840aa660 con 0x7f09840b0a00 2023-11-02T15:34:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840ac520 con 0x7f098413ede0 2023-11-02T15:34:00.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:00.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098b29d640 10 monclient(hunting): authenticate will time out at 1367.614713s 2023-11-02T15:34:00.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098983a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 0x7f09841d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0989039640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09840b0a00 0x7f09840ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38730/0 (socket says 172.21.15.174:38730) 2023-11-02T15:34:00.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 1 -- 172.21.15.174:0/1962445914 learned_addr learned my addr 172.21.15.174:0/1962445914 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:00.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 10 monclient(hunting): get_auth_request con 0x7f09840b3c10 auth_method 0 2023-11-02T15:34:00.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:00.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:00.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f0988838640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:00.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098983a640 10 monclient(hunting): get_auth_request con 0x7f098413ede0 auth_method 0 2023-11-02T15:34:00.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098983a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:00.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098983a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.272+0000 7f098983a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): get_auth_request con 0x7f09840b0a00 auth_method 0 2023-11-02T15:34:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:00.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:00.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:00.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:00.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0988838640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:00.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f098983a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:00.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f098983a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:00.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f098983a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:00.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:00.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:00.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.273+0000 7f0989039640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:00.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 10 monclient(hunting): handle_auth_done global_id 4260 payload 824 2023-11-02T15:34:00.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 -- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 msgr2=0x7f09841d6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 0x7f09841d6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 -- 172.21.15.174:0/1962445914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09840b0a00 msgr2=0x7f09840ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:00.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09840b0a00 0x7f09840ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:00.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 10 monclient: _finish_hunting 0 2023-11-02T15:34:00.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 monclient: found mon.noname-a 2023-11-02T15:34:00.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:00.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f0988838640 1 -- 172.21.15.174:0/1962445914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740094d0 con 0x7f09840b3c10 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.276+0000 7f098983a640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 0x7f09841d6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.279+0000 7f0988838640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09740090b0 tx=0x7f09740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.279+0000 7f0979ffb640 10 client.?.objecter ms_handle_connect 0x7f09840b3c10 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.279+0000 7f0979ffb640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097403abe0 con 0x7f09840b3c10 2023-11-02T15:34:00.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: dump: 2023-11-02T15:34:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.317 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.317 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:00.317 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.317 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:00.317 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:00.318 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:00.318 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: _finish_auth 0 2023-11-02T15:34:00.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:34:00.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:00.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097403ad40 con 0x7f09840b3c10 2023-11-02T15:34:00.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:00.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974044630 con 0x7f09840b3c10 2023-11-02T15:34:00.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:00.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:00.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f0979ffb640 10 monclient: dump: 2023-11-02T15:34:00.320 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:00.320 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:00.321 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.321 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:00.321 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:00.322 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:00.322 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:00.322 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:00.322 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:00.322 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:00.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f098b29d640 5 monclient: authenticate success, global_id 4260 2023-11-02T15:34:00.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f097b7fe640 4 set_mon_vals no callback set 2023-11-02T15:34:00.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f098b29d640 10 monclient: _renew_subs 2023-11-02T15:34:00.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f098b29d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.282+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09840a8720 con 0x7f09840b3c10 2023-11-02T15:34:00.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.283+0000 7f098b29d640 10 client.4260.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:00.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.283+0000 7f098b29d640 10 monclient: _renew_subs 2023-11-02T15:34:00.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.283+0000 7f098b29d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.283+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09841d7180 con 0x7f09840b3c10 2023-11-02T15:34:00.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.283+0000 7f098b29d640 1 librados: init done 2023-11-02T15:34:00.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.284+0000 7f098b29d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:00.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.284+0000 7f098b29d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:00.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.284+0000 7f098b29d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.284+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09840ac6e0 con 0x7f09840b3c10 2023-11-02T15:34:00.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.304+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f097404d450 con 0x7f09840b3c10 2023-11-02T15:34:00.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.304+0000 7f0979ffb640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 0x7f096806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:00.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.304+0000 7f0989039640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 0x7f096806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:00.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.304+0000 7f0989039640 10 monclient: get_auth_request con 0x7f096806be10 auth_method 0 2023-11-02T15:34:00.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.305+0000 7f0989039640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 0x7f096806e2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f097c009b40 tx=0x7f097c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:00.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.305+0000 7f0979ffb640 10 client.4260.objecter ms_handle_connect 0x7f096806be10 2023-11-02T15:34:00.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.307+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f097402d040 con 0x7f09840b3c10 2023-11-02T15:34:00.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.307+0000 7f0979ffb640 10 client.4260.objecter ms_dispatch 0x7f0984000b90 osd_map(20..20 src has 1..20) v4 2023-11-02T15:34:00.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.307+0000 7f0979ffb640 3 client.4260.objecter handle_osd_map got epochs [20,20] > 0 2023-11-02T15:34:00.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.307+0000 7f0979ffb640 3 client.4260.objecter handle_osd_map decoding full epoch 20 2023-11-02T15:34:00.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.307+0000 7f0979ffb640 20 client.4260.objecter dump_active .. 0 homeless 2023-11-02T15:34:00.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.581+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f09740764f0 con 0x7f09840b3c10 2023-11-02T15:34:00.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.581+0000 7f0979ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:00.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.581+0000 7f0979ffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:00.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.777+0000 7f098b29d640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-11-02T15:34:00.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.777+0000 7f098b29d640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-11-02T15:34:00.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.777+0000 7f098b29d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:00.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:00.777+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09840ac6e0 con 0x7f09840b3c10 2023-11-02T15:34:01.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.171+0000 7f0979ffb640 1 -- 172.21.15.174:0/1962445914 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v21) v1 ==== 173+0+0 (secure 0 0 0) 0x7f09840ac6e0 con 0x7f09840b3c10 2023-11-02T15:34:01.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.171+0000 7f0979ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2023-11-02T15:34:01.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.171+0000 7f0979ffb640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2023-11-02T15:34:01.174 INFO:teuthology.orchestra.run.smithi174.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-02T15:34:01.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 10 librados: watch_flush enter 2023-11-02T15:34:01.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 10 librados: watch_flush exit 2023-11-02T15:34:01.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 20 client.4260.objecter shutdown clearing up homeless session... 2023-11-02T15:34:01.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 10 client.4260.objecter successfully canceled tick 2023-11-02T15:34:01.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 msgr2=0x7f096806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 0x7f096806e2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f097c009b40 tx=0x7f097c009190 comp rx=0 tx=0).stop 2023-11-02T15:34:01.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 10 monclient: shutdown 2023-11-02T15:34:01.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:01.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 msgr2=0x7f09840abfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09740090b0 tx=0x7f09740046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 shutdown_connections 2023-11-02T15:34:01.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f096806be10 0x7f096806e2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f098413ede0 0x7f09841d6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09840b3c10 0x7f09840abfe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 --2- 172.21.15.174:0/1962445914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09840b0a00 0x7f09840ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 >> 172.21.15.174:0/1962445914 conn(0x7f0984065960 msgr2=0x7f09840b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:01.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 shutdown_connections 2023-11-02T15:34:01.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 -- 172.21.15.174:0/1962445914 wait complete. 2023-11-02T15:34:01.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.173+0000 7f098b29d640 1 librados: shutdown 2023-11-02T15:34:01.187 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-02T15:34:01.188 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-02T15:34:01.188 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-02T15:34:01.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-02T15:34:01.189 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-02T15:34:01.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.289+0000 7f510e269640 10 monclient: get_monmap_and_config 2023-11-02T15:34:01.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.289+0000 7f510e269640 10 monclient: build_initial_monmap 2023-11-02T15:34:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.290+0000 7f510e269640 10 monclient: monmap: 2023-11-02T15:34:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:01.291911+0000 2023-11-02T15:34:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:01.291911+0000 2023-11-02T15:34:01.294 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:01.294 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.294 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:01.294 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:01.294 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:01.295 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:01.295 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:01.295 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.290+0000 7f510e269640 1 Processor -- start 2023-11-02T15:34:01.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.290+0000 7f510e269640 1 -- start start 2023-11-02T15:34:01.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.290+0000 7f510e269640 10 monclient: init 2023-11-02T15:34:01.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:01.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2023-11-02T15:34:01.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): picked mon.noname-b con 0x7f51080b4170 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:01.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f51080a8df0 0x7f5108141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): picked mon.noname-c con 0x7f51080a8df0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:01.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5108142280 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:01.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080aa3d0 con 0x7f5108142280 2023-11-02T15:34:01.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:01.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510813f420 con 0x7f51080a8df0 2023-11-02T15:34:01.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:01.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b4550 con 0x7f51080b4170 2023-11-02T15:34:01.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:01.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.291+0000 7f510e269640 10 monclient(hunting): authenticate will time out at 1368.634213s 2023-11-02T15:34:01.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:01.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f51077fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f51080a8df0 0x7f5108141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:01.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38808/0 (socket says 172.21.15.174:38808) 2023-11-02T15:34:01.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 1 -- 172.21.15.174:0/2424268325 learned_addr learned my addr 172.21.15.174:0/2424268325 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:01.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 10 monclient(hunting): get_auth_request con 0x7f51080b4170 auth_method 0 2023-11-02T15:34:01.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:01.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:01.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.292+0000 7f5107fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:01.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:01.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:01.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:01.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f51077fe640 10 monclient(hunting): get_auth_request con 0x7f51080a8df0 auth_method 0 2023-11-02T15:34:01.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f51077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f51077fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.293+0000 7f51077fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.294+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.294+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.294+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 10 monclient(hunting): handle_auth_done global_id 4235 payload 274 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 1 -- 172.21.15.174:0/2424268325 >> v2:172.21.15.175:3300/0 conn(0x7f51080a8df0 msgr2=0x7f5108141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 1 --2- 172.21.15.174:0/2424268325 >> v2:172.21.15.175:3300/0 conn(0x7f51080a8df0 0x7f5108141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 1 -- 172.21.15.174:0/2424268325 >> v1:172.21.15.175:6789/0 conn(0x7f5108142280 legacy=0x7f51080b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:01.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 1 monclient: found mon.noname-b 2023-11-02T15:34:01.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:01.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:01.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.295+0000 7f5107fff640 1 -- 172.21.15.174:0/2424268325 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080a92d0 con 0x7f51080b4170 2023-11-02T15:34:01.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.298+0000 7f5107fff640 1 --2- 172.21.15.174:0/2424268325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5dd99f797db837e server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 1 -- 172.21.15.174:0/2424268325 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8031640 con 0x7f51080b4170 2023-11-02T15:34:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:01.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: dump: 2023-11-02T15:34:01.309 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:01.309 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:01.309 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.309 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.309 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:01.310 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:01.310 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 1 -- 172.21.15.174:0/2424268325 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f80317a0 con 0x7f51080b4170 2023-11-02T15:34:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 1 -- 172.21.15.174:0/2424268325 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8031b30 con 0x7f51080b4170 2023-11-02T15:34:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f5106ffd640 10 monclient: dump: 2023-11-02T15:34:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:01.313 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:01.313 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.313 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.313 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:01.313 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.314 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:01.314 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:01.314 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:01.314 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f510e269640 5 monclient: authenticate success, global_id 4235 2023-11-02T15:34:01.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f510e269640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:01.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.304+0000 7f510e269640 4 set_mon_vals no callback set 2023-11-02T15:34:01.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 10 monclient: shutdown 2023-11-02T15:34:01.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:01.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 -- 172.21.15.174:0/2424268325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 msgr2=0x7f5108141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 --2- 172.21.15.174:0/2424268325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f802b8f0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 -- 172.21.15.174:0/2424268325 shutdown_connections 2023-11-02T15:34:01.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 --2- 172.21.15.174:0/2424268325 >> v2:172.21.15.175:3300/0 conn(0x7f51080a8df0 0x7f5108141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 --2- 172.21.15.174:0/2424268325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f5108141800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 -- 172.21.15.174:0/2424268325 >> 172.21.15.174:0/2424268325 conn(0x7f5108065d80 msgr2=0x7f5108066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:01.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 -- 172.21.15.174:0/2424268325 shutdown_connections 2023-11-02T15:34:01.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.305+0000 7f510e269640 1 -- 172.21.15.174:0/2424268325 wait complete. 2023-11-02T15:34:01.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.306+0000 7f510e269640 10 monclient: build_initial_monmap 2023-11-02T15:34:01.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.306+0000 7f510e269640 10 monclient: monmap: 2023-11-02T15:34:01.325 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:01.325 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:01.325 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:01.325 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:01.325 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:01.326 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.326 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:01.326 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:01.326 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:01.326 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.306+0000 7f510e269640 1 librados: starting msgr at 2023-11-02T15:34:01.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.306+0000 7f510e269640 1 librados: starting objecter 2023-11-02T15:34:01.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.306+0000 7f510e269640 1 Processor -- start 2023-11-02T15:34:01.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 -- start start 2023-11-02T15:34:01.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 librados: setting wanted keys 2023-11-02T15:34:01.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 librados: calling monclient init 2023-11-02T15:34:01.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient: init 2023-11-02T15:34:01.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:01.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:34:01.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): picked mon.noname-b con 0x7f51080a8df0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:01.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f51080ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): picked mon.noname-c con 0x7f51080b4170 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:01.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108142280 0x7f51080abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): picked mon.noname-a con 0x7f5108142280 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:01.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:01.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081d9b30 con 0x7f5108142280 2023-11-02T15:34:01.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:01.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080ac150 con 0x7f51080a8df0 2023-11-02T15:34:01.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:01.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5108145240 con 0x7f51080b4170 2023-11-02T15:34:01.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:01.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.307+0000 7f510e269640 10 monclient(hunting): authenticate will time out at 1368.650455s 2023-11-02T15:34:01.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:01.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f51077fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f51080ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:01.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57802/0 (socket says 172.21.15.174:57802) 2023-11-02T15:34:01.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 1 -- 172.21.15.174:0/521377469 learned_addr learned my addr 172.21.15.174:0/521377469 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:01.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 10 monclient(hunting): get_auth_request con 0x7f51080a8df0 auth_method 0 2023-11-02T15:34:01.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:01.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f5107fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f51077fe640 10 monclient(hunting): get_auth_request con 0x7f51080b4170 auth_method 0 2023-11-02T15:34:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f51077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f51077fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:01.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.308+0000 7f51077fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f51077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:01.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.309+0000 7f5107fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.313+0000 7f5107fff640 10 monclient(hunting): handle_auth_done global_id 4261 payload 824 2023-11-02T15:34:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.313+0000 7f5107fff640 1 -- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 msgr2=0x7f51080ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.313+0000 7f5107fff640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f51080ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.313+0000 7f5107fff640 1 -- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108142280 msgr2=0x7f51080abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.313+0000 7f5107fff640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108142280 0x7f51080abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.314+0000 7f5107fff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.314+0000 7f5107fff640 1 monclient: found mon.noname-b 2023-11-02T15:34:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.314+0000 7f5107fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:01.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.314+0000 7f5107fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.314+0000 7f5107fff640 1 -- 172.21.15.174:0/521377469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80094d0 con 0x7f51080a8df0 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.315+0000 7f5107fff640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.315+0000 7f5104ff9640 10 client.?.objecter ms_handle_connect 0x7f51080a8df0 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.315+0000 7f5104ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f803ac60 con 0x7f51080a8df0 2023-11-02T15:34:01.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:01.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:01.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: dump: 2023-11-02T15:34:01.338 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:01.338 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:01.338 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.339 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.339 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:01.339 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.339 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:01.339 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:01.340 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:01.340 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: _finish_auth 0 2023-11-02T15:34:01.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:34:01.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f803adc0 con 0x7f51080a8df0 2023-11-02T15:34:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8044630 con 0x7f51080a8df0 2023-11-02T15:34:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f5104ff9640 10 monclient: dump: 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:01.342 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:01.343 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:01.343 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:01.343 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:01.343 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:01.343 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:01.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 5 monclient: authenticate success, global_id 4261 2023-11-02T15:34:01.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 10 monclient: _renew_subs 2023-11-02T15:34:01.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:01.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51081436e0 con 0x7f51080a8df0 2023-11-02T15:34:01.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f51067fc640 4 set_mon_vals no callback set 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 10 client.4261.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 10 monclient: _renew_subs 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081da590 con 0x7f51080a8df0 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.319+0000 7f510e269640 1 librados: init done 2023-11-02T15:34:01.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.320+0000 7f510e269640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:01.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.320+0000 7f510e269640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:01.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.320+0000 7f510e269640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:01.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.320+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51081d9990 con 0x7f51080a8df0 2023-11-02T15:34:01.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.340+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f50f802f050 con 0x7f51080a8df0 2023-11-02T15:34:01.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.340+0000 7f5104ff9640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 0x7f50f406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:01.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.340+0000 7f51077fe640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 0x7f50f406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:01.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.340+0000 7f51077fe640 10 monclient: get_auth_request con 0x7f50f406bdc0 auth_method 0 2023-11-02T15:34:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.341+0000 7f51077fe640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 0x7f50f406e280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5108066380 tx=0x7f50fc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.341+0000 7f5104ff9640 10 client.4261.objecter ms_handle_connect 0x7f50f406bdc0 2023-11-02T15:34:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.343+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f50f807a520 con 0x7f51080a8df0 2023-11-02T15:34:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.343+0000 7f5104ff9640 10 client.4261.objecter ms_dispatch 0x7f5108000b90 osd_map(21..21 src has 1..21) v4 2023-11-02T15:34:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.343+0000 7f5104ff9640 3 client.4261.objecter handle_osd_map got epochs [21,21] > 0 2023-11-02T15:34:01.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.343+0000 7f5104ff9640 3 client.4261.objecter handle_osd_map decoding full epoch 21 2023-11-02T15:34:01.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.343+0000 7f5104ff9640 20 client.4261.objecter dump_active .. 0 homeless 2023-11-02T15:34:01.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.607+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f50f8076160 con 0x7f51080a8df0 2023-11-02T15:34:01.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.607+0000 7f5104ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:01.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.607+0000 7f5104ff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:01.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.869+0000 7f510e269640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2023-11-02T15:34:01.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.869+0000 7f510e269640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2023-11-02T15:34:01.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.869+0000 7f510e269640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:01.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:01.869+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f51081d9990 con 0x7f51080a8df0 2023-11-02T15:34:02.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.071+0000 7f5104ff9640 1 -- 172.21.15.174:0/521377469 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169969 (secure 0 0 0) 0x7f51081d9990 con 0x7f51080a8df0 2023-11-02T15:34:02.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.071+0000 7f5104ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2023-11-02T15:34:02.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.071+0000 7f5104ff9640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:02.076 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:02.078 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 10 librados: watch_flush enter 2023-11-02T15:34:02.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 10 librados: watch_flush exit 2023-11-02T15:34:02.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 20 client.4261.objecter shutdown clearing up homeless session... 2023-11-02T15:34:02.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 10 client.4261.objecter successfully canceled tick 2023-11-02T15:34:02.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 msgr2=0x7f50f406e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 0x7f50f406e280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5108066380 tx=0x7f50fc009190 comp rx=0 tx=0).stop 2023-11-02T15:34:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 10 monclient: shutdown 2023-11-02T15:34:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 msgr2=0x7f51081da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f80046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 shutdown_connections 2023-11-02T15:34:02.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108142280 0x7f51080abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f50f406bdc0 0x7f50f406e280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51080b4170 0x7f51080ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 --2- 172.21.15.174:0/521377469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51080a8df0 0x7f51081da050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 >> 172.21.15.174:0/521377469 conn(0x7f5108065d80 msgr2=0x7f51080b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:02.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 shutdown_connections 2023-11-02T15:34:02.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 -- 172.21.15.174:0/521377469 wait complete. 2023-11-02T15:34:02.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.073+0000 7f510e269640 1 librados: shutdown 2023-11-02T15:34:02.088 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":85844},{"type":"v1","addr":"172.21.15.174:6801","nonce":85844}]},"active_addr":"172.21.15.174:6801/85844","active_change":"2023-11-02T15:33:10.515734+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":443284825}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2263031913}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":4009488067}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2331373194}]}]} 2023-11-02T15:34:02.098 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-02T15:34:02.099 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-02T15:34:02.099 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-02T15:34:02.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.200+0000 7fdf58f5a640 10 monclient: get_monmap_and_config 2023-11-02T15:34:02.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.200+0000 7fdf58f5a640 10 monclient: build_initial_monmap 2023-11-02T15:34:02.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.200+0000 7fdf58f5a640 10 monclient: monmap: 2023-11-02T15:34:02.204 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:02.204 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.204 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:02.202709+0000 2023-11-02T15:34:02.204 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:02.202709+0000 2023-11-02T15:34:02.204 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:02.205 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.205 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:02.205 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:02.205 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:02.205 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:02.206 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:02.206 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.201+0000 7fdf58f5a640 1 Processor -- start 2023-11-02T15:34:02.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.201+0000 7fdf58f5a640 1 -- start start 2023-11-02T15:34:02.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.201+0000 7fdf58f5a640 10 monclient: init 2023-11-02T15:34:02.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:02.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2023-11-02T15:34:02.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf540b4260 0x7fdf540a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-c con 0x7fdf540b4260 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:02.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdf54141730 0x7fdf54141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fdf54141730 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:02.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdf540b1a00 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:02.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540a9210 con 0x7fdf540b1a00 2023-11-02T15:34:02.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540aa550 con 0x7fdf54141730 2023-11-02T15:34:02.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540b4640 con 0x7fdf540b4260 2023-11-02T15:34:02.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:02.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.202+0000 7fdf58f5a640 10 monclient(hunting): authenticate will time out at 1369.545021s 2023-11-02T15:34:02.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf540b4260 0x7fdf540a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:02.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf540b4260 0x7fdf540a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57806/0 (socket says 172.21.15.174:57806) 2023-11-02T15:34:02.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 1 -- 172.21.15.174:0/1803829797 learned_addr learned my addr 172.21.15.174:0/1803829797 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:02.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 10 monclient(hunting): get_auth_request con 0x7fdf540b4260 auth_method 0 2023-11-02T15:34:02.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:02.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:02.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.204+0000 7fdf5259c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:02.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.205+0000 7fdf5259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:02.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.205+0000 7fdf5259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:02.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.205+0000 7fdf5259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:02.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 10 monclient(hunting): handle_auth_done global_id 4264 payload 274 2023-11-02T15:34:02.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 1 -- 172.21.15.174:0/1803829797 >> v2:172.21.15.174:3300/0 conn(0x7fdf54141730 msgr2=0x7fdf54141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:02.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 1 --2- 172.21.15.174:0/1803829797 >> v2:172.21.15.174:3300/0 conn(0x7fdf54141730 0x7fdf54141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 1 -- 172.21.15.174:0/1803829797 >> v1:172.21.15.175:6789/0 conn(0x7fdf540b1a00 legacy=0x7fdf540b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 10 monclient: _finish_hunting 0 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 1 monclient: found mon.noname-c 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.207+0000 7fdf5259c640 1 -- 172.21.15.174:0/1803829797 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540aa750 con 0x7fdf540b4260 2023-11-02T15:34:02.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.211+0000 7fdf5259c640 1 --2- 172.21.15.174:0/1803829797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf540b4260 0x7fdf540a8870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009870 tx=0x7fdf4402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f397d6151db32f server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:02.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.215+0000 7fdf5159a640 1 -- 172.21.15.174:0/1803829797 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44031870 con 0x7fdf540b4260 2023-11-02T15:34:02.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.215+0000 7fdf5159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:02.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:02.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: dump: 2023-11-02T15:34:02.220 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:02.220 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.220 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.220 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.221 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:02.221 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.221 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:02.221 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:02.221 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:02.222 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: _finish_auth 0 2023-11-02T15:34:02.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: _check_auth_tickets 2023-11-02T15:34:02.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:02.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 1 -- 172.21.15.174:0/1803829797 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf440319d0 con 0x7fdf540b4260 2023-11-02T15:34:02.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:02.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 1 -- 172.21.15.174:0/1803829797 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44031d60 con 0x7fdf540b4260 2023-11-02T15:34:02.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 5 monclient: authenticate success, global_id 4264 2023-11-02T15:34:02.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:02.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 4 set_mon_vals no callback set 2023-11-02T15:34:02.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:02.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:02.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf5159a640 10 monclient: dump: 2023-11-02T15:34:02.224 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:02.224 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.225 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.225 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.225 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:02.225 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.225 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:02.226 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:02.226 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:02.226 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 10 monclient: shutdown 2023-11-02T15:34:02.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 -- 172.21.15.174:0/1803829797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf540b4260 msgr2=0x7fdf540a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/1803829797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf540b4260 0x7fdf540a8870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009870 tx=0x7fdf4402b910 comp rx=0 tx=0).stop 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 -- 172.21.15.174:0/1803829797 shutdown_connections 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/1803829797 >> v2:172.21.15.174:3300/0 conn(0x7fdf54141730 0x7fdf54141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/1803829797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf540b4260 0x7fdf540a8870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 -- 172.21.15.174:0/1803829797 >> 172.21.15.174:0/1803829797 conn(0x7fdf54065d50 msgr2=0x7fdf54066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:02.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 -- 172.21.15.174:0/1803829797 shutdown_connections 2023-11-02T15:34:02.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.216+0000 7fdf58f5a640 1 -- 172.21.15.174:0/1803829797 wait complete. 2023-11-02T15:34:02.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.217+0000 7fdf58f5a640 10 monclient: build_initial_monmap 2023-11-02T15:34:02.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.217+0000 7fdf58f5a640 10 monclient: monmap: 2023-11-02T15:34:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:02.233 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.233 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:02.233 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:02.233 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:02.233 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.217+0000 7fdf58f5a640 1 librados: starting msgr at 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.217+0000 7fdf58f5a640 1 librados: starting objecter 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 Processor -- start 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 -- start start 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 librados: setting wanted keys 2023-11-02T15:34:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 librados: calling monclient init 2023-11-02T15:34:02.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient: init 2023-11-02T15:34:02.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:02.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:02.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf540b1a00 0x7fdf540aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-c con 0x7fdf540b1a00 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:02.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf540b4260 0x7fdf5413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fdf540b4260 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:02.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): picked mon.noname-b con 0x7fdf54141730 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:02.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf541d6830 con 0x7fdf540b4260 2023-11-02T15:34:02.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5413e5c0 con 0x7fdf54141730 2023-11-02T15:34:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5413f290 con 0x7fdf540b1a00 2023-11-02T15:34:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.218+0000 7fdf58f5a640 10 monclient(hunting): authenticate will time out at 1369.561493s 2023-11-02T15:34:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57810/0 (socket says 172.21.15.174:57810) 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 1 -- 172.21.15.174:0/2987435712 learned_addr learned my addr 172.21.15.174:0/2987435712 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 10 monclient(hunting): get_auth_request con 0x7fdf54141730 auth_method 0 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.219+0000 7fdf52d9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:02.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.220+0000 7fdf52d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:02.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.220+0000 7fdf52d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:02.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.220+0000 7fdf52d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:02.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.221+0000 7fdf51d9b640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf540b4260 0x7fdf5413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:02.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.221+0000 7fdf51d9b640 10 monclient(hunting): get_auth_request con 0x7fdf540b4260 auth_method 0 2023-11-02T15:34:02.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.221+0000 7fdf51d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:02.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.221+0000 7fdf51d9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:02.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.221+0000 7fdf51d9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:02.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.222+0000 7fdf5259c640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf540b1a00 0x7fdf540aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:02.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 10 monclient(hunting): handle_auth_done global_id 4267 payload 824 2023-11-02T15:34:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 -- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf540b1a00 msgr2=0x7fdf540aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf540b1a00 0x7fdf540aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 -- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf540b4260 msgr2=0x7fdf5413ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf540b4260 0x7fdf5413ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 10 monclient: _finish_hunting 0 2023-11-02T15:34:02.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 monclient: found mon.noname-b 2023-11-02T15:34:02.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:02.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.223+0000 7fdf52d9d640 1 -- 172.21.15.174:0/2987435712 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440094d0 con 0x7fdf54141730 2023-11-02T15:34:02.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.227+0000 7fdf52d9d640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c7a0 tx=0x7fdf4c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:02.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.227+0000 7fdf42ffd640 10 client.?.objecter ms_handle_connect 0x7fdf54141730 2023-11-02T15:34:02.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.227+0000 7fdf42ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:02.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003b70 con 0x7fdf54141730 2023-11-02T15:34:02.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:02.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:02.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 10 monclient: dump: 2023-11-02T15:34:02.288 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:02.288 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.288 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.288 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.288 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:02.289 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.290 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:02.290 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:02.290 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:02.290 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:02.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:02.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.283+0000 7fdf42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:02.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c003cf0 con 0x7fdf54141730 2023-11-02T15:34:02.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 5 monclient: authenticate success, global_id 4267 2023-11-02T15:34:02.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 monclient: _renew_subs 2023-11-02T15:34:02.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5413c270 con 0x7fdf54141730 2023-11-02T15:34:02.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:02.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003ea0 con 0x7fdf54141730 2023-11-02T15:34:02.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf42ffd640 10 monclient: dump: 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.294 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:02.295 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:02.295 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.295 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:02.295 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:02.295 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:02.296 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf50d99640 4 set_mon_vals no callback set 2023-11-02T15:34:02.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 client.4267.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:02.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 monclient: _renew_subs 2023-11-02T15:34:02.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf541d7450 con 0x7fdf54141730 2023-11-02T15:34:02.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 1 librados: init done 2023-11-02T15:34:02.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.284+0000 7fdf58f5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:02.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.285+0000 7fdf58f5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:02.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.285+0000 7fdf58f5a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.285+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54067920 con 0x7fdf54141730 2023-11-02T15:34:02.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.549+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdf4c00aa90 con 0x7fdf54141730 2023-11-02T15:34:02.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.550+0000 7fdf42ffd640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 0x7fdf3406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.550+0000 7fdf5259c640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 0x7fdf3406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:02.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.550+0000 7fdf5259c640 10 monclient: get_auth_request con 0x7fdf3406be40 auth_method 0 2023-11-02T15:34:02.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.550+0000 7fdf42ffd640 10 client.4267.objecter ms_handle_connect 0x7fdf3406be40 2023-11-02T15:34:02.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.550+0000 7fdf5259c640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 0x7fdf3406e300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009870 tx=0x7fdf4403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:02.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.552+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdf4c058580 con 0x7fdf54141730 2023-11-02T15:34:02.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.552+0000 7fdf42ffd640 10 client.4267.objecter ms_dispatch 0x7fdf54000b90 osd_map(21..21 src has 1..21) v4 2023-11-02T15:34:02.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.552+0000 7fdf42ffd640 3 client.4267.objecter handle_osd_map got epochs [21,21] > 0 2023-11-02T15:34:02.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.552+0000 7fdf42ffd640 3 client.4267.objecter handle_osd_map decoding full epoch 21 2023-11-02T15:34:02.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.552+0000 7fdf42ffd640 20 client.4267.objecter dump_active .. 0 homeless 2023-11-02T15:34:02.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.567+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdf4c0548c0 con 0x7fdf54141730 2023-11-02T15:34:02.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.567+0000 7fdf42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:02.570 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.567+0000 7fdf42ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:02.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.739+0000 7fdf58f5a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:02.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.739+0000 7fdf58f5a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:02.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.739+0000 7fdf58f5a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:02.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.739+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf540b54a0 con 0x7fdf54141730 2023-11-02T15:34:02.841 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.839+0000 7fdf42ffd640 1 -- 172.21.15.174:0/2987435712 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+14242 (secure 0 0 0) 0x7fdf4c057ee0 con 0x7fdf54141730 2023-11-02T15:34:02.841 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.839+0000 7fdf42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:02.841 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.839+0000 7fdf42ffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:02.842 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:02.842 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":22,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:34:02.203616+0000","last_up_change":"2023-11-02T15:33:53.161912+0000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-02T15:33:53.130231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-02T15:33:59.744250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":86149},{"type":"v1","addr":"172.21.15.174:6803","nonce":86149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":86149},{"type":"v1","addr":"172.21.15.174:6805","nonce":86149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":86149},{"type":"v1","addr":"172.21.15.174:6809","nonce":86149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":86149},{"type":"v1","addr":"172.21.15.174:6807","nonce":86149}]},"public_addr":"172.21.15.174:6803/86149","cluster_addr":"172.21.15.174:6805/86149","heartbeat_back_addr":"172.21.15.174:6809/86149","heartbeat_front_addr":"172.21.15.174:6807/86149","state":["exists","up"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":86146},{"type":"v1","addr":"172.21.15.174:6821","nonce":86146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":86146},{"type":"v1","addr":"172.21.15.174:6827","nonce":86146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":86146},{"type":"v1","addr":"172.21.15.174:6833","nonce":86146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":86146},{"type":"v1","addr":"172.21.15.174:6831","nonce":86146}]},"public_addr":"172.21.15.174:6821/86146","cluster_addr":"172.21.15.174:6827/86146","heartbeat_back_addr":"172.21.15.174:6833/86146","heartbeat_front_addr":"172.21.15.174:6831/86146","state":["exists","up"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":86153},{"type":"v1","addr":"172.21.15.174:6811","nonce":86153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":86153},{"type":"v1","addr":"172.21.15.174:6813","nonce":86153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":86153},{"type":"v1","addr":"172.21.15.174:6817","nonce":86153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":86153},{"type":"v1","addr":"172.21.15.174:6815","nonce":86153}]},"public_addr":"172.21.15.174:6811/86153","cluster_addr":"172.21.15.174:6813/86153","heartbeat_back_addr":"172.21.15.174:6817/86153","heartbeat_front_addr":"172.21.15.174:6815/86153","state":["exists","up"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":86148},{"type":"v1","addr":"172.21.15.174:6820","nonce":86148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":86148},{"type":"v1","addr":"172.21.15.174:6823","nonce":86148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":86148},{"type":"v1","addr":"172.21.15.174:6829","nonce":86148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":86148},{"type":"v1","addr":"172.21.15.174:6826","nonce":86148}]},"public_addr":"172.21.15.174:6820/86148","cluster_addr":"172.21.15.174:6823/86148","heartbeat_back_addr":"172.21.15.174:6829/86148","heartbeat_front_addr":"172.21.15.174:6826/86148","state":["exists","up"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":86415},{"type":"v1","addr":"172.21.15.175:6817","nonce":86415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":86415},{"type":"v1","addr":"172.21.15.175:6819","nonce":86415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":86415},{"type":"v1","addr":"172.21.15.175:6823","nonce":86415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":86415},{"type":"v1","addr":"172.21.15.175:6821","nonce":86415}]},"public_addr":"172.21.15.175:6817/86415","cluster_addr":"172.21.15.175:6819/86415","heartbeat_back_addr":"172.21.15.175:6823/86415","heartbeat_front_addr":"172.21.15.175:6821/86415","state":["exists","up"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":86416},{"type":"v1","addr":"172.21.15.175:6809","nonce":86416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":86416},{"type":"v1","addr":"172.21.15.175:6811","nonce":86416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":86416},{"type":"v1","addr":"172.21.15.175:6815","nonce":86416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":86416},{"type":"v1","addr":"172.21.15.175:6813","nonce":86416}]},"public_addr":"172.21.15.175:6809/86416","cluster_addr":"172.21.15.175:6811/86416","heartbeat_back_addr":"172.21.15.175:6815/86416","heartbeat_front_addr":"172.21.15.175:6813/86416","state":["exists","up"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":86417},{"type":"v1","addr":"172.21.15.175:6825","nonce":86417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":86417},{"type":"v1","addr":"172.21.15.175:6827","nonce":86417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":86417},{"type":"v1","addr":"172.21.15.175:6831","nonce":86417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":86417},{"type":"v1","addr":"172.21.15.175:6829","nonce":86417}]},"public_addr":"172.21.15.175:6825/86417","cluster_addr":"172.21.15.175:6827/86417","heartbeat_back_addr":"172.21.15.175:6831/86417","heartbeat_front_addr":"172.21.15.175:6829/86417","state":["exists","up"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":86418},{"type":"v1","addr":"172.21.15.175:6801","nonce":86418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":86418},{"type":"v1","addr":"172.21.15.175:6803","nonce":86418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":86418},{"type":"v1","addr":"172.21.15.175:6807","nonce":86418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":86418},{"type":"v1","addr":"172.21.15.175:6805","nonce":86418}]},"public_addr":"172.21.15.175:6801/86418","cluster_addr":"172.21.15.175:6803/86418","heartbeat_back_addr":"172.21.15.175:6807/86418","heartbeat_front_addr":"172.21.15.175:6805/86418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.105568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:49.744420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.135477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.453210+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.626045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.898023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.470083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:47.661882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:34:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 10 librados: watch_flush enter 2023-11-02T15:34:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 10 librados: watch_flush exit 2023-11-02T15:34:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 20 client.4267.objecter shutdown clearing up homeless session... 2023-11-02T15:34:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 10 client.4267.objecter successfully canceled tick 2023-11-02T15:34:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 msgr2=0x7fdf3406e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 0x7fdf3406e300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009870 tx=0x7fdf4403c040 comp rx=0 tx=0).stop 2023-11-02T15:34:02.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 10 monclient: shutdown 2023-11-02T15:34:02.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:02.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 msgr2=0x7fdf541d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:02.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c7a0 tx=0x7fdf4c00cc60 comp rx=0 tx=0).stop 2023-11-02T15:34:02.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 shutdown_connections 2023-11-02T15:34:02.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf54141730 0x7fdf541d6d30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf540b4260 0x7fdf5413ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fdf3406be40 0x7fdf3406e300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 --2- 172.21.15.174:0/2987435712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf540b1a00 0x7fdf540aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:02.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 >> 172.21.15.174:0/2987435712 conn(0x7fdf54065d50 msgr2=0x7fdf54066130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:02.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.840+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 shutdown_connections 2023-11-02T15:34:02.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.841+0000 7fdf58f5a640 1 -- 172.21.15.174:0/2987435712 wait complete. 2023-11-02T15:34:02.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.841+0000 7fdf58f5a640 1 librados: shutdown 2023-11-02T15:34:02.854 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-02T15:34:02.855 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-02T15:34:02.941 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:15.095 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:02.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.954+0000 7f4c9c077640 10 monclient: get_monmap_and_config 2023-11-02T15:34:02.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.954+0000 7f4c9c077640 10 monclient: build_initial_monmap 2023-11-02T15:34:02.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.954+0000 7f4c9c077640 10 monclient: monmap: 2023-11-02T15:34:02.957 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:02.957 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:02.956209+0000 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:02.956209+0000 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:02.958 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:02.959 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:02.959 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:02.959 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:02.959 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:02.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.954+0000 7f4c9c077640 1 Processor -- start 2023-11-02T15:34:02.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 -- start start 2023-11-02T15:34:02.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient: init 2023-11-02T15:34:02.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:02.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2023-11-02T15:34:02.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c940b4260 0x7f4c940a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-c con 0x7f4c940b4260 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:02.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4c94141730 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:02.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 0x7f4c9413e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:02.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-b con 0x7f4c9413e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:02.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c940a9210 con 0x7f4c94141730 2023-11-02T15:34:02.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940aa550 con 0x7f4c940b4260 2023-11-02T15:34:02.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:02.965 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b4640 con 0x7f4c9413e520 2023-11-02T15:34:02.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:02.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:02.955+0000 7f4c9c077640 10 monclient(hunting): authenticate will time out at 1370.298389s 2023-11-02T15:34:02.998 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:15.152 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.032+0000 7f4c99e13640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c940b4260 0x7f4c940a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:03.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:15.188 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.032+0000 7f4c9a614640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 0x7f4c9413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:03.036 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:15.131 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.034+0000 7f4c99612640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4c94141730 0x7f4c94141ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:38584/0 (socket says 172.21.15.174:38584) 2023-11-02T15:34:03.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.034+0000 7f4c99612640 1 -- 172.21.15.174:0/358263290 learned_addr learned my addr 172.21.15.174:0/358263290 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:03.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.036+0000 7f4c9a614640 10 monclient(hunting): get_auth_request con 0x7f4c9413e520 auth_method 0 2023-11-02T15:34:03.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.036+0000 7f4c9a614640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:03.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.036+0000 7f4c9a614640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:03.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.036+0000 7f4c9a614640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:03.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.053+0000 7f4c99e13640 10 monclient(hunting): get_auth_request con 0x7f4c940b4260 auth_method 0 2023-11-02T15:34:03.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.053+0000 7f4c99e13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:03.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.053+0000 7f4c99e13640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:03.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.053+0000 7f4c99e13640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:03.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.079+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:03.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.079+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:03.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.079+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:03.092 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:15.185 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 10 monclient(hunting): handle_auth_done global_id 4241 payload 274 2023-11-02T15:34:03.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 1 -- 172.21.15.174:0/358263290 >> v2:172.21.15.175:3300/0 conn(0x7f4c940b4260 msgr2=0x7f4c940a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:03.093 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:15.246 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 1 --2- 172.21.15.174:0/358263290 >> v2:172.21.15.175:3300/0 conn(0x7f4c940b4260 0x7f4c940a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:03.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 1 -- 172.21.15.174:0/358263290 >> v1:172.21.15.175:6789/0 conn(0x7f4c94141730 legacy=0x7f4c94141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:03.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 10 monclient: _finish_hunting 0 2023-11-02T15:34:03.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 1 monclient: found mon.noname-b 2023-11-02T15:34:03.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:03.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:03.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.090+0000 7f4c9a614640 1 -- 172.21.15.174:0/358263290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c941422b0 con 0x7f4c9413e520 2023-11-02T15:34:03.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.194+0000 7f4c9a614640 1 --2- 172.21.15.174:0/358263290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 0x7f4c9413e940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009dc0 tx=0x7f4c8c02be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193b32a59607afdf server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:03.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 1 -- 172.21.15.174:0/358263290 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c02cbb0 con 0x7f4c9413e520 2023-11-02T15:34:03.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:03.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: dump: 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.232 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:03.233 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:03.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: _finish_auth 0 2023-11-02T15:34:03.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: _check_auth_tickets 2023-11-02T15:34:03.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:03.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 1 -- 172.21.15.174:0/358263290 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c02cd10 con 0x7f4c9413e520 2023-11-02T15:34:03.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 1 -- 172.21.15.174:0/358263290 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c034660 con 0x7f4c9413e520 2023-11-02T15:34:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c9c077640 5 monclient: authenticate success, global_id 4241 2023-11-02T15:34:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c9c077640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c9c077640 4 set_mon_vals no callback set 2023-11-02T15:34:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.228+0000 7f4c98e11640 10 monclient: dump: 2023-11-02T15:34:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:03.242 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.243 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.243 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:03.243 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:03.243 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:03.244 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:03.244 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:03.244 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:03.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 10 monclient: shutdown 2023-11-02T15:34:03.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:03.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 -- 172.21.15.174:0/358263290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 msgr2=0x7f4c9413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:03.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 --2- 172.21.15.174:0/358263290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 0x7f4c9413e940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009dc0 tx=0x7f4c8c02be00 comp rx=0 tx=0).stop 2023-11-02T15:34:03.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 -- 172.21.15.174:0/358263290 shutdown_connections 2023-11-02T15:34:03.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 --2- 172.21.15.174:0/358263290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c9413e520 0x7f4c9413e940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:03.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 --2- 172.21.15.174:0/358263290 >> v2:172.21.15.175:3300/0 conn(0x7f4c940b4260 0x7f4c940a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:03.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 -- 172.21.15.174:0/358263290 >> 172.21.15.174:0/358263290 conn(0x7f4c94065d50 msgr2=0x7f4c94066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:03.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 -- 172.21.15.174:0/358263290 shutdown_connections 2023-11-02T15:34:03.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.229+0000 7f4c9c077640 1 -- 172.21.15.174:0/358263290 wait complete. 2023-11-02T15:34:03.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.230+0000 7f4c9c077640 10 monclient: build_initial_monmap 2023-11-02T15:34:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.230+0000 7f4c9c077640 10 monclient: monmap: 2023-11-02T15:34:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:03.251 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:03.251 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:03.251 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:03.251 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:03.251 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:03.252 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:03.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 librados: starting msgr at 2023-11-02T15:34:03.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 librados: starting objecter 2023-11-02T15:34:03.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 Processor -- start 2023-11-02T15:34:03.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 -- start start 2023-11-02T15:34:03.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 librados: setting wanted keys 2023-11-02T15:34:03.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 1 librados: calling monclient init 2023-11-02T15:34:03.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.231+0000 7f4c9c077640 10 monclient: init 2023-11-02T15:34:03.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:03.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:03.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c940b4260 0x7f4c941cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:03.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-c con 0x7f4c940b4260 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:03.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9413e520 0x7f4c941d6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:03.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-b con 0x7f4c9413e520 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:03.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:03.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): picked mon.noname-a con 0x7f4c94141730 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:03.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:03.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9413db00 con 0x7f4c94141730 2023-11-02T15:34:03.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:03.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9413c560 con 0x7f4c9413e520 2023-11-02T15:34:03.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:03.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b0b70 con 0x7f4c940b4260 2023-11-02T15:34:03.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:03.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.232+0000 7f4c9c077640 10 monclient(hunting): authenticate will time out at 1370.574889s 2023-11-02T15:34:03.257 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:15.410 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.281+0000 7f4c9a614640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:03.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38762/0 (socket says 172.21.15.174:38762) 2023-11-02T15:34:03.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 1 -- 172.21.15.174:0/2606960028 learned_addr learned my addr 172.21.15.174:0/2606960028 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:03.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 10 monclient(hunting): get_auth_request con 0x7f4c94141730 auth_method 0 2023-11-02T15:34:03.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:03.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:03.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.283+0000 7f4c9a614640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:03.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.286+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:03.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.286+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:03.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.286+0000 7f4c9a614640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:03.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 10 monclient(hunting): handle_auth_done global_id 4266 payload 824 2023-11-02T15:34:03.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 -- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c940b4260 msgr2=0x7f4c941cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:03.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c940b4260 0x7f4c941cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:03.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 -- 172.21.15.174:0/2606960028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9413e520 msgr2=0x7f4c941d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:03.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9413e520 0x7f4c941d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:03.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 10 monclient: _finish_hunting 0 2023-11-02T15:34:03.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 monclient: found mon.noname-a 2023-11-02T15:34:03.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:03.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:03.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.291+0000 7f4c9a614640 1 -- 172.21.15.174:0/2606960028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009a70 con 0x7f4c94141730 2023-11-02T15:34:03.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.309+0000 7f4c9a614640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ee0 tx=0x7f4c8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:03.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.309+0000 7f4c8a7fc640 10 client.?.objecter ms_handle_connect 0x7f4c94141730 2023-11-02T15:34:03.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.309+0000 7f4c8a7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:03.332 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:15.428 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.340 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:15.493 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:03.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.397+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c040650 con 0x7f4c94141730 2023-11-02T15:34:03.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.397+0000 7f4c8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:03.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:03.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: dump: 2023-11-02T15:34:03.401 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:03.402 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:03.402 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.402 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.402 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:03.403 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:03.403 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:03.403 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:03.403 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:03.403 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:03.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:03.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:03.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:03.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0407d0 con 0x7f4c94141730 2023-11-02T15:34:03.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:03.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c040b80 con 0x7f4c94141730 2023-11-02T15:34:03.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 5 monclient: authenticate success, global_id 4266 2023-11-02T15:34:03.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 10 monclient: _renew_subs 2023-11-02T15:34:03.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:03.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c941d6b60 con 0x7f4c94141730 2023-11-02T15:34:03.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8bfff640 4 set_mon_vals no callback set 2023-11-02T15:34:03.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:03.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:03.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c8a7fc640 10 monclient: dump: 2023-11-02T15:34:03.406 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:03.407 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:03.407 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.407 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:03.407 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:03.408 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:03.408 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:03.408 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:03.408 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:03.408 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:03.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 10 client.4266.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:03.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 10 monclient: _renew_subs 2023-11-02T15:34:03.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:03.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9413e170 con 0x7f4c94141730 2023-11-02T15:34:03.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.398+0000 7f4c9c077640 1 librados: init done 2023-11-02T15:34:03.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.399+0000 7f4c9c077640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:03.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.399+0000 7f4c9c077640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:03.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.399+0000 7f4c9c077640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:03.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.399+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94067920 con 0x7f4c94141730 2023-11-02T15:34:03.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.679+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4c8c040ce0 con 0x7f4c94141730 2023-11-02T15:34:03.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.679+0000 7f4c8a7fc640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 0x7f4c7806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:03.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.679+0000 7f4c99e13640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 0x7f4c7806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:03.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.679+0000 7f4c99e13640 10 monclient: get_auth_request con 0x7f4c7806be10 auth_method 0 2023-11-02T15:34:03.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.680+0000 7f4c8a7fc640 10 client.4266.objecter ms_handle_connect 0x7f4c7806be10 2023-11-02T15:34:03.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.680+0000 7f4c99e13640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 0x7f4c7806e2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098f0 tx=0x7f4c90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:03.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4c8c07b850 con 0x7f4c94141730 2023-11-02T15:34:03.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 10 client.4266.objecter ms_dispatch 0x7f4c94000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:03.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 3 client.4266.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:03.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 3 client.4266.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:03.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 20 client.4266.objecter dump_active .. 0 homeless 2023-11-02T15:34:03.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4c8c077490 con 0x7f4c94141730 2023-11-02T15:34:03.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:03.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.693+0000 7f4c8a7fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:03.898 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.896+0000 7f4c9c077640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:03.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.896+0000 7f4c9c077640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:03.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.896+0000 7f4c9c077640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:03.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:03.896+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c940b0d30 con 0x7f4c94141730 2023-11-02T15:34:04.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.009+0000 7f4c8a7fc640 1 -- 172.21.15.174:0/2606960028 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+14242 (secure 0 0 0) 0x7f4c940b0d30 con 0x7f4c94141730 2023-11-02T15:34:04.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.009+0000 7f4c8a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T15:34:04.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.009+0000 7f4c8a7fc640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:04.013 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:04.013 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":22,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T15:34:02.203616+0000","last_up_change":"2023-11-02T15:33:53.161912+0000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-02T15:33:53.130231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-02T15:33:59.744250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":86149},{"type":"v1","addr":"172.21.15.174:6803","nonce":86149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":86149},{"type":"v1","addr":"172.21.15.174:6805","nonce":86149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":86149},{"type":"v1","addr":"172.21.15.174:6809","nonce":86149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":86149},{"type":"v1","addr":"172.21.15.174:6807","nonce":86149}]},"public_addr":"172.21.15.174:6803/86149","cluster_addr":"172.21.15.174:6805/86149","heartbeat_back_addr":"172.21.15.174:6809/86149","heartbeat_front_addr":"172.21.15.174:6807/86149","state":["exists","up"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":86146},{"type":"v1","addr":"172.21.15.174:6821","nonce":86146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":86146},{"type":"v1","addr":"172.21.15.174:6827","nonce":86146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":86146},{"type":"v1","addr":"172.21.15.174:6833","nonce":86146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":86146},{"type":"v1","addr":"172.21.15.174:6831","nonce":86146}]},"public_addr":"172.21.15.174:6821/86146","cluster_addr":"172.21.15.174:6827/86146","heartbeat_back_addr":"172.21.15.174:6833/86146","heartbeat_front_addr":"172.21.15.174:6831/86146","state":["exists","up"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":86153},{"type":"v1","addr":"172.21.15.174:6811","nonce":86153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":86153},{"type":"v1","addr":"172.21.15.174:6813","nonce":86153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":86153},{"type":"v1","addr":"172.21.15.174:6817","nonce":86153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":86153},{"type":"v1","addr":"172.21.15.174:6815","nonce":86153}]},"public_addr":"172.21.15.174:6811/86153","cluster_addr":"172.21.15.174:6813/86153","heartbeat_back_addr":"172.21.15.174:6817/86153","heartbeat_front_addr":"172.21.15.174:6815/86153","state":["exists","up"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":86148},{"type":"v1","addr":"172.21.15.174:6820","nonce":86148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":86148},{"type":"v1","addr":"172.21.15.174:6823","nonce":86148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":86148},{"type":"v1","addr":"172.21.15.174:6829","nonce":86148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":86148},{"type":"v1","addr":"172.21.15.174:6826","nonce":86148}]},"public_addr":"172.21.15.174:6820/86148","cluster_addr":"172.21.15.174:6823/86148","heartbeat_back_addr":"172.21.15.174:6829/86148","heartbeat_front_addr":"172.21.15.174:6826/86148","state":["exists","up"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":86415},{"type":"v1","addr":"172.21.15.175:6817","nonce":86415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":86415},{"type":"v1","addr":"172.21.15.175:6819","nonce":86415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":86415},{"type":"v1","addr":"172.21.15.175:6823","nonce":86415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":86415},{"type":"v1","addr":"172.21.15.175:6821","nonce":86415}]},"public_addr":"172.21.15.175:6817/86415","cluster_addr":"172.21.15.175:6819/86415","heartbeat_back_addr":"172.21.15.175:6823/86415","heartbeat_front_addr":"172.21.15.175:6821/86415","state":["exists","up"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":86416},{"type":"v1","addr":"172.21.15.175:6809","nonce":86416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":86416},{"type":"v1","addr":"172.21.15.175:6811","nonce":86416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":86416},{"type":"v1","addr":"172.21.15.175:6815","nonce":86416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":86416},{"type":"v1","addr":"172.21.15.175:6813","nonce":86416}]},"public_addr":"172.21.15.175:6809/86416","cluster_addr":"172.21.15.175:6811/86416","heartbeat_back_addr":"172.21.15.175:6815/86416","heartbeat_front_addr":"172.21.15.175:6813/86416","state":["exists","up"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":86417},{"type":"v1","addr":"172.21.15.175:6825","nonce":86417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":86417},{"type":"v1","addr":"172.21.15.175:6827","nonce":86417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":86417},{"type":"v1","addr":"172.21.15.175:6831","nonce":86417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":86417},{"type":"v1","addr":"172.21.15.175:6829","nonce":86417}]},"public_addr":"172.21.15.175:6825/86417","cluster_addr":"172.21.15.175:6827/86417","heartbeat_back_addr":"172.21.15.175:6831/86417","heartbeat_front_addr":"172.21.15.175:6829/86417","state":["exists","up"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":86418},{"type":"v1","addr":"172.21.15.175:6801","nonce":86418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":86418},{"type":"v1","addr":"172.21.15.175:6803","nonce":86418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":86418},{"type":"v1","addr":"172.21.15.175:6807","nonce":86418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":86418},{"type":"v1","addr":"172.21.15.175:6805","nonce":86418}]},"public_addr":"172.21.15.175:6801/86418","cluster_addr":"172.21.15.175:6803/86418","heartbeat_back_addr":"172.21.15.175:6807/86418","heartbeat_front_addr":"172.21.15.175:6805/86418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.105568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:49.744420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.135477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.453210+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.626045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.898023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.470083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:47.661882+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T15:34:04.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.011+0000 7f4c9c077640 10 librados: watch_flush enter 2023-11-02T15:34:04.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.011+0000 7f4c9c077640 10 librados: watch_flush exit 2023-11-02T15:34:04.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.011+0000 7f4c9c077640 20 client.4266.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.011+0000 7f4c9c077640 10 client.4266.objecter successfully canceled tick 2023-11-02T15:34:04.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 msgr2=0x7f4c7806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 0x7f4c7806e2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098f0 tx=0x7f4c90008040 comp rx=0 tx=0).stop 2023-11-02T15:34:04.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 10 monclient: shutdown 2023-11-02T15:34:04.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 msgr2=0x7f4c94140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ee0 tx=0x7f4c8c004a40 comp rx=0 tx=0).stop 2023-11-02T15:34:04.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 shutdown_connections 2023-11-02T15:34:04.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94141730 0x7f4c94140680 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9413e520 0x7f4c941d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f4c7806be10 0x7f4c7806e2d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 --2- 172.21.15.174:0/2606960028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c940b4260 0x7f4c941cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 >> 172.21.15.174:0/2606960028 conn(0x7f4c94065d50 msgr2=0x7f4c9413b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 shutdown_connections 2023-11-02T15:34:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 -- 172.21.15.174:0/2606960028 wait complete. 2023-11-02T15:34:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.012+0000 7f4c9c077640 1 librados: shutdown 2023-11-02T15:34:04.026 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-02T15:34:04.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-02T15:34:04.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-02T15:34:04.028 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-02T15:34:04.029 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-02T15:34:04.029 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-02T15:34:04.030 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-02T15:34:04.031 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-02T15:34:04.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.213+0000 7f924359e640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.213+0000 7f924359e640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.213+0000 7f924359e640 10 monclient: monmap: 2023-11-02T15:34:04.218 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.218 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.218 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.215502+0000 2023-11-02T15:34:04.218 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.215502+0000 2023-11-02T15:34:04.219 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.219 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.219 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.219 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.219 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.220 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.220 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.220 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.214+0000 7f575d6cc640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.214+0000 7f575d6cc640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.214+0000 7f575d6cc640 10 monclient: monmap: 2023-11-02T15:34:04.222 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.222 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.223 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.216421+0000 2023-11-02T15:34:04.223 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.216421+0000 2023-11-02T15:34:04.223 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.223 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.224 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.224 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.225 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.225 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.225 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.214+0000 7f924359e640 1 Processor -- start 2023-11-02T15:34:04.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.216+0000 7f575d6cc640 1 Processor -- start 2023-11-02T15:34:04.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.216+0000 7f575d6cc640 1 -- start start 2023-11-02T15:34:04.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.216+0000 7f575d6cc640 10 monclient: init 2023-11-02T15:34:04.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2023-11-02T15:34:04.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57580a9eb0 0x7f57580aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-c con 0x7f57580a9eb0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:04.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f57580aa290 0x7f57580abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-a con 0x7f57580aa290 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:04.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57580a9470 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:04.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57580a8f90 con 0x7f57580a9470 2023-11-02T15:34:04.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57580aae60 con 0x7f57580aa290 2023-11-02T15:34:04.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57580ab040 con 0x7f57580a9eb0 2023-11-02T15:34:04.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f575d6cc640 10 monclient(hunting): authenticate will time out at 1371.560614s 2023-11-02T15:34:04.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.215+0000 7f924359e640 1 -- start start 2023-11-02T15:34:04.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.215+0000 7f924359e640 10 monclient: init 2023-11-02T15:34:04.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient: _add_conns ranks=[1,4,3,2,0] 2023-11-02T15:34:04.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f923c034d70 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:04.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f923c0c3c80 0x7f923c0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): picked mon.noname-c con 0x7f923c0c3c80 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:04.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 0x7f923c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): picked mon.noname-a con 0x7f923c0cc580 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c02cd40 con 0x7f923c034d70 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c02cb40 con 0x7f923c0cc580 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c035150 con 0x7f923c0c3c80 2023-11-02T15:34:04.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.217+0000 7f924359e640 10 monclient(hunting): authenticate will time out at 1371.559781s 2023-11-02T15:34:04.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57580a9eb0 0x7f57580aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57580a9eb0 0x7f57580aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57844/0 (socket says 172.21.15.174:57844) 2023-11-02T15:34:04.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 1 -- 172.21.15.174:0/1450108826 learned_addr learned my addr 172.21.15.174:0/1450108826 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 10 monclient(hunting): get_auth_request con 0x7f57580a9eb0 auth_method 0 2023-11-02T15:34:04.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.219+0000 7f5757fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f57577fe640 1 --2- 172.21.15.174:0/1450108826 >> v2:172.21.15.174:3300/0 conn(0x7f57580aa290 0x7f57580abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f57577fe640 10 monclient(hunting): get_auth_request con 0x7f57580aa290 auth_method 0 2023-11-02T15:34:04.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f57577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f57577fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f57577fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 0x7f923c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 0x7f923c0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38764/0 (socket says 172.21.15.174:38764) 2023-11-02T15:34:04.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 1 -- 172.21.15.174:0/208242714 learned_addr learned my addr 172.21.15.174:0/208242714 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 10 monclient(hunting): get_auth_request con 0x7f923c0cc580 auth_method 0 2023-11-02T15:34:04.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.220+0000 7f9242d9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f9241d9b640 1 --2- 172.21.15.174:0/208242714 >> v2:172.21.15.175:3300/0 conn(0x7f923c0c3c80 0x7f923c0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9242d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9242d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9242d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9241d9b640 10 monclient(hunting): get_auth_request con 0x7f923c0c3c80 auth_method 0 2023-11-02T15:34:04.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9241d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9241d9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9241d9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f954fd0a640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f954fd0a640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.222+0000 7f954fd0a640 10 monclient: monmap: 2023-11-02T15:34:04.244 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.245 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.245 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.224508+0000 2023-11-02T15:34:04.245 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.224508+0000 2023-11-02T15:34:04.246 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.246 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.246 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.246 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.247 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.247 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.247 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.247 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f954fd0a640 1 Processor -- start 2023-11-02T15:34:04.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 10 monclient(hunting): handle_auth_done global_id 4279 payload 274 2023-11-02T15:34:04.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 1 -- 172.21.15.174:0/208242714 >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 msgr2=0x7f923c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 1 --2- 172.21.15.174:0/208242714 >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 0x7f923c0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f954fd0a640 1 -- start start 2023-11-02T15:34:04.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f954fd0a640 10 monclient: init 2023-11-02T15:34:04.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 1 -- 172.21.15.174:0/208242714 >> v1:172.21.15.175:6789/0 conn(0x7f923c034d70 legacy=0x7f923c0326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 1 monclient: found mon.noname-c 2023-11-02T15:34:04.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.225+0000 7f9241d9b640 1 -- 172.21.15.174:0/208242714 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0c17c0 con 0x7f923c0c3c80 2023-11-02T15:34:04.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2023-11-02T15:34:04.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-b con 0x7f95480a9730 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f954813abc0 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:04.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f954813b000 0x7f95480ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-a con 0x7f954813b000 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:04.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480aa460 con 0x7f954813abc0 2023-11-02T15:34:04.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480a9b10 con 0x7f954813b000 2023-11-02T15:34:04.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480af430 con 0x7f95480a9730 2023-11-02T15:34:04.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.227+0000 7f954fd0a640 10 monclient(hunting): authenticate will time out at 1371.569798s 2023-11-02T15:34:04.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38860/0 (socket says 172.21.15.174:38860) 2023-11-02T15:34:04.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 1 -- 172.21.15.174:0/3977946388 learned_addr learned my addr 172.21.15.174:0/3977946388 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 10 monclient(hunting): get_auth_request con 0x7f95480a9730 auth_method 0 2023-11-02T15:34:04.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.228+0000 7f954daa6640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.230+0000 7f954daa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.230+0000 7f954daa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.230+0000 7f954daa6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f5757fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f5757fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.224+0000 7f5757fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 10 monclient(hunting): handle_auth_done global_id 4276 payload 274 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 1 -- 172.21.15.174:0/1450108826 >> v2:172.21.15.174:3300/0 conn(0x7f57580aa290 msgr2=0x7f57580abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 1 --2- 172.21.15.174:0/1450108826 >> v2:172.21.15.174:3300/0 conn(0x7f57580aa290 0x7f57580abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 1 -- 172.21.15.174:0/1450108826 >> v1:172.21.15.174:6789/0 conn(0x7f57580a9470 legacy=0x7f5758140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 1 monclient: found mon.noname-c 2023-11-02T15:34:04.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.229+0000 7f5757fff640 1 -- 172.21.15.174:0/1450108826 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580ab320 con 0x7f57580a9eb0 2023-11-02T15:34:04.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f85e21dd640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f85e21dd640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.223+0000 7f85e21dd640 10 monclient: monmap: 2023-11-02T15:34:04.261 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.261 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.261 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.225391+0000 2023-11-02T15:34:04.262 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.225391+0000 2023-11-02T15:34:04.262 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.262 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.262 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.262 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.265 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.268 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.268 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.268 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.231+0000 7f85e21dd640 1 Processor -- start 2023-11-02T15:34:04.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.234+0000 7f9241d9b640 1 --2- 172.21.15.174:0/208242714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c0c3c80 0x7f923c0c40f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f923c00ba40 tx=0x7f923402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d68a9420edf5b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 10 monclient(hunting): handle_auth_done global_id 4244 payload 274 2023-11-02T15:34:04.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 1 -- 172.21.15.174:0/3977946388 >> v2:172.21.15.174:3300/0 conn(0x7f954813b000 msgr2=0x7f95480ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 1 --2- 172.21.15.174:0/3977946388 >> v2:172.21.15.174:3300/0 conn(0x7f954813b000 0x7f95480ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 1 -- 172.21.15.174:0/3977946388 >> v1:172.21.15.174:6789/0 conn(0x7f954813abc0 legacy=0x7f95480abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f954daa6640 1 -- 172.21.15.174:0/3977946388 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954813b490 con 0x7f95480a9730 2023-11-02T15:34:04.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.234+0000 7f85e21dd640 1 -- start start 2023-11-02T15:34:04.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.234+0000 7f85e21dd640 10 monclient: init 2023-11-02T15:34:04.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient: _add_conns ranks=[1,0,2,4,3] 2023-11-02T15:34:04.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f85dc0a9730 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:04.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f85dc0aeef0 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:04.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-b con 0x7f85dc0ac560 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc0aa460 con 0x7f85dc0aeef0 2023-11-02T15:34:04.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc0ac9f0 con 0x7f85dc0a9730 2023-11-02T15:34:04.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0a9b10 con 0x7f85dc0ac560 2023-11-02T15:34:04.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.235+0000 7f85e21dd640 10 monclient(hunting): authenticate will time out at 1371.578280s 2023-11-02T15:34:04.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38866/0 (socket says 172.21.15.174:38866) 2023-11-02T15:34:04.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 1 -- 172.21.15.174:0/3061828950 learned_addr learned my addr 172.21.15.174:0/3061828950 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 10 monclient(hunting): get_auth_request con 0x7f85dc0ac560 auth_method 0 2023-11-02T15:34:04.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.236+0000 7f85dbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.237+0000 7f85dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.237+0000 7f85dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.237+0000 7f85dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 1 -- 172.21.15.174:0/208242714 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234031640 con 0x7f923c0c3c80 2023-11-02T15:34:04.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:04.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: dump: 2023-11-02T15:34:04.282 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.282 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.282 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.284 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.284 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 1 -- 172.21.15.174:0/208242714 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92340317a0 con 0x7f923c0c3c80 2023-11-02T15:34:04.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 1 -- 172.21.15.174:0/208242714 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234031b10 con 0x7f923c0c3c80 2023-11-02T15:34:04.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:04.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.238+0000 7f924159a640 10 monclient: dump: 2023-11-02T15:34:04.286 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.287 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.287 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.287 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.287 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.287 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.290 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.291 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.291 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 5 monclient: authenticate success, global_id 4279 2023-11-02T15:34:04.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 4 set_mon_vals no callback set 2023-11-02T15:34:04.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 10 monclient: shutdown 2023-11-02T15:34:04.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 1 -- 172.21.15.174:0/208242714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c0c3c80 msgr2=0x7f923c0c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7f924359e640 1 --2- 172.21.15.174:0/208242714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c0c3c80 0x7f923c0c40f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f923c00ba40 tx=0x7f923402b8f0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7f924359e640 1 -- 172.21.15.174:0/208242714 shutdown_connections 2023-11-02T15:34:04.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7f924359e640 1 --2- 172.21.15.174:0/208242714 >> v2:172.21.15.174:3300/0 conn(0x7f923c0cc580 0x7f923c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7f924359e640 1 --2- 172.21.15.174:0/208242714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c0c3c80 0x7f923c0c40f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7f924359e640 1 -- 172.21.15.174:0/208242714 >> 172.21.15.174:0/208242714 conn(0x7f923c0061c0 msgr2=0x7f923c0065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7ff55e9a4640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7ff55e9a4640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.239+0000 7ff55e9a4640 10 monclient: monmap: 2023-11-02T15:34:04.294 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.295 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.295 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.241328+0000 2023-11-02T15:34:04.295 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.241328+0000 2023-11-02T15:34:04.295 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.295 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.296 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.296 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.296 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.296 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.296 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.297 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7f5757fff640 1 --2- 172.21.15.174:0/1450108826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f57580aba20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57580aef50 tx=0x7f575002e360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38798f128794430 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7ff55e9a4640 1 Processor -- start 2023-11-02T15:34:04.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7ff55e9a4640 1 -- start start 2023-11-02T15:34:04.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.240+0000 7ff55e9a4640 10 monclient: init 2023-11-02T15:34:04.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7f924359e640 1 -- 172.21.15.174:0/208242714 shutdown_connections 2023-11-02T15:34:04.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2023-11-02T15:34:04.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff55806a940 0x7ff5580aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-c con 0x7ff55806a940 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:04.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff5580a94c0 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:04.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-b con 0x7ff5580abe50 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580aa460 con 0x7ff5580a94c0 2023-11-02T15:34:04.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580af430 con 0x7ff55806a940 2023-11-02T15:34:04.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55813b2d0 con 0x7ff5580abe50 2023-11-02T15:34:04.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7ff55e9a4640 10 monclient(hunting): authenticate will time out at 1371.584565s 2023-11-02T15:34:04.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.241+0000 7f924359e640 1 -- 172.21.15.174:0/208242714 wait complete. 2023-11-02T15:34:04.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.242+0000 7f924359e640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.242+0000 7f924359e640 10 monclient: monmap: 2023-11-02T15:34:04.307 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.307 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.307 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.307 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.308 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.308 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.308 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.308 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.309 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.309 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 librados: starting msgr at 2023-11-02T15:34:04.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 librados: starting objecter 2023-11-02T15:34:04.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 Processor -- start 2023-11-02T15:34:04.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 -- start start 2023-11-02T15:34:04.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 10 monclient(hunting): handle_auth_done global_id 4247 payload 274 2023-11-02T15:34:04.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 1 -- 172.21.15.174:0/3061828950 >> v1:172.21.15.175:6789/0 conn(0x7f85dc0a9730 legacy=0x7f85dc13abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 1 -- 172.21.15.174:0/3061828950 >> v1:172.21.15.174:6789/0 conn(0x7f85dc0aeef0 legacy=0x7f85dc0abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f85dbfff640 1 -- 172.21.15.174:0/3061828950 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0af330 con 0x7f85dc0ac560 2023-11-02T15:34:04.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 librados: setting wanted keys 2023-11-02T15:34:04.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 1 librados: calling monclient init 2023-11-02T15:34:04.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.243+0000 7f924359e640 10 monclient: init 2023-11-02T15:34:04.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:04.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c034d70 0x7f923c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38872/0 (socket says 172.21.15.174:38872) 2023-11-02T15:34:04.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 1 -- 172.21.15.174:0/3025194639 learned_addr learned my addr 172.21.15.174:0/3025194639 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 10 monclient(hunting): get_auth_request con 0x7ff5580abe50 auth_method 0 2023-11-02T15:34:04.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7ff55cf41640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): picked mon.noname-b con 0x7f923c034d70 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 0x7f923c0c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): picked mon.noname-a con 0x7f923c0c3c80 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f923c0cc580 0x7f923c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): picked mon.noname-c con 0x7f923c0cc580 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0c1210 con 0x7f923c0c3c80 2023-11-02T15:34:04.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ca1d0 con 0x7f923c034d70 2023-11-02T15:34:04.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0cbf50 con 0x7f923c0cc580 2023-11-02T15:34:04.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.244+0000 7f924359e640 10 monclient(hunting): authenticate will time out at 1371.586982s 2023-11-02T15:34:04.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.245+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.245+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.245+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.246+0000 7f6801f05640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.246+0000 7f6801f05640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.246+0000 7f6801f05640 10 monclient: monmap: 2023-11-02T15:34:04.324 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.325 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.325 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.248594+0000 2023-11-02T15:34:04.325 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.248594+0000 2023-11-02T15:34:04.325 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.325 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.326 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.326 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.326 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.326 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.326 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.327 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f416ec42640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f416ec42640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f416ec42640 10 monclient: monmap: 2023-11-02T15:34:04.329 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.329 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.329 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.249073+0000 2023-11-02T15:34:04.329 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.249073+0000 2023-11-02T15:34:04.329 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.330 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.330 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.330 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.331 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.331 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.332 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.332 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7ff557fff640 1 --2- 172.21.15.174:0/3025194639 >> v2:172.21.15.175:3300/0 conn(0x7ff55806a940 0x7ff5580aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f6801f05640 1 Processor -- start 2023-11-02T15:34:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f6801f05640 1 -- start start 2023-11-02T15:34:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f6801f05640 10 monclient: init 2023-11-02T15:34:04.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.247+0000 7f416ec42640 1 Processor -- start 2023-11-02T15:34:04.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 1 -- start start 2023-11-02T15:34:04.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 10 monclient: init 2023-11-02T15:34:04.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 0x7f923c0c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f6801f05640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f6801f05640 10 monclient: _add_conns ranks=[1,2,4,3,0] 2023-11-02T15:34:04.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 0x7f923c0c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38790/0 (socket says 172.21.15.174:38790) 2023-11-02T15:34:04.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 1 -- 172.21.15.174:0/2686787306 learned_addr learned my addr 172.21.15.174:0/2686787306 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 10 monclient(hunting): get_auth_request con 0x7f923c0c3c80 auth_method 0 2023-11-02T15:34:04.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f67fc0b3b70 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:04.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f9241d9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2023-11-02T15:34:04.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f41680a84b0 0x7f4168144a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f6801f05640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-c con 0x7f41680a84b0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:04.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.248+0000 7f416ec42640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f41680a8890 0x7f41680b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-b con 0x7f67fc0b3220 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-a con 0x7f41680a8890 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:04.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f41680b0ec0 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:04.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680b7780 con 0x7f41680b0ec0 2023-11-02T15:34:04.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680b7a10 con 0x7f41680a8890 2023-11-02T15:34:04.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680a9440 con 0x7f41680a84b0 2023-11-02T15:34:04.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67fc13ef10 0x7f67fc144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f416ec42640 10 monclient(hunting): authenticate will time out at 1371.591779s 2023-11-02T15:34:04.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-c con 0x7f67fc13ef10 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:04.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff557fff640 10 monclient(hunting): get_auth_request con 0x7ff55806a940 auth_method 0 2023-11-02T15:34:04.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff557fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff557fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff557fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc0a9fd0 con 0x7f67fc0b3b70 2023-11-02T15:34:04.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc0b3f50 con 0x7f67fc13ef10 2023-11-02T15:34:04.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc0a9200 con 0x7f67fc0b3220 2023-11-02T15:34:04.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f6801f05640 10 monclient(hunting): authenticate will time out at 1371.592069s 2023-11-02T15:34:04.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 10 monclient(hunting): handle_auth_done global_id 4250 payload 274 2023-11-02T15:34:04.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 1 -- 172.21.15.174:0/3025194639 >> v2:172.21.15.175:3300/0 conn(0x7ff55806a940 msgr2=0x7ff5580aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 1 --2- 172.21.15.174:0/3025194639 >> v2:172.21.15.175:3300/0 conn(0x7ff55806a940 0x7ff5580aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 1 -- 172.21.15.174:0/3025194639 >> v1:172.21.15.175:6789/0 conn(0x7ff5580a94c0 legacy=0x7ff55813abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7ff55cf41640 1 -- 172.21.15.174:0/3025194639 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580aa8f0 con 0x7ff5580abe50 2023-11-02T15:34:04.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.249+0000 7f9241d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f41680a8890 0x7f41680b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 1 -- 172.21.15.174:0/1450108826 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750003b90 con 0x7f57580a9eb0 2023-11-02T15:34:04.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:04.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: dump: 2023-11-02T15:34:04.355 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.355 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.355 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.356 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.357 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 1 -- 172.21.15.174:0/1450108826 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750032070 con 0x7f57580a9eb0 2023-11-02T15:34:04.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 1 -- 172.21.15.174:0/1450108826 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750004030 con 0x7f57580a9eb0 2023-11-02T15:34:04.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:04.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f5756ffd640 10 monclient: dump: 2023-11-02T15:34:04.359 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.359 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.359 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.359 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.361 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f575d6cc640 5 monclient: authenticate success, global_id 4276 2023-11-02T15:34:04.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f575d6cc640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f575d6cc640 4 set_mon_vals no callback set 2023-11-02T15:34:04.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f41680a8890 0x7f41680b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38802/0 (socket says 172.21.15.174:38802) 2023-11-02T15:34:04.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 1 -- 172.21.15.174:0/589037048 learned_addr learned my addr 172.21.15.174:0/589037048 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 10 monclient(hunting): get_auth_request con 0x7f41680a8890 auth_method 0 2023-11-02T15:34:04.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.250+0000 7f416d43f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 10 monclient: shutdown 2023-11-02T15:34:04.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 -- 172.21.15.174:0/1450108826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 msgr2=0x7f57580aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 --2- 172.21.15.174:0/1450108826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f57580aba20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57580aef50 tx=0x7f575002e360 comp rx=0 tx=0).stop 2023-11-02T15:34:04.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 -- 172.21.15.174:0/1450108826 shutdown_connections 2023-11-02T15:34:04.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 --2- 172.21.15.174:0/1450108826 >> v2:172.21.15.174:3300/0 conn(0x7f57580aa290 0x7f57580abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 --2- 172.21.15.174:0/1450108826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f57580aba20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 -- 172.21.15.174:0/1450108826 >> 172.21.15.174:0/1450108826 conn(0x7f57580af4d0 msgr2=0x7f57580af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 -- 172.21.15.174:0/1450108826 shutdown_connections 2023-11-02T15:34:04.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.251+0000 7f575d6cc640 1 -- 172.21.15.174:0/1450108826 wait complete. 2023-11-02T15:34:04.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 10 monclient: monmap: 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.367 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.368 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.368 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.368 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.368 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.368 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 librados: starting msgr at 2023-11-02T15:34:04.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 librados: starting objecter 2023-11-02T15:34:04.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 Processor -- start 2023-11-02T15:34:04.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f9242d9d640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f923c0cc580 0x7f923c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.370 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 -- start start 2023-11-02T15:34:04.370 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 librados: setting wanted keys 2023-11-02T15:34:04.370 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 1 librados: calling monclient init 2023-11-02T15:34:04.371 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f575d6cc640 10 monclient: init 2023-11-02T15:34:04.371 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.252+0000 7f85dbfff640 1 --2- 172.21.15.174:0/3061828950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85d400d3a0 tx=0x7f85d402e310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc956aecf9062ce server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.371 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:04.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57580a9470 0x7f57581369e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-c con 0x7f57580a9470 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f575813c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-b con 0x7f57580a9eb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57580aa290 0x7f57581c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): picked mon.noname-a con 0x7f57580aa290 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57581c5560 con 0x7f57580aa290 2023-11-02T15:34:04.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57581c3a70 con 0x7f57580a9eb0 2023-11-02T15:34:04.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575813ba30 con 0x7f57580a9470 2023-11-02T15:34:04.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f575d6cc640 10 monclient(hunting): authenticate will time out at 1371.596168s 2023-11-02T15:34:04.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2023-11-02T15:34:04.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 -- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f923c0cc580 msgr2=0x7f923c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f923c0cc580 0x7f923c0ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 -- 172.21.15.174:0/2686787306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c034d70 msgr2=0x7f923c0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f416dc40640 1 --2- 172.21.15.174:0/589037048 >> v2:172.21.15.175:3300/0 conn(0x7f41680a84b0 0x7f4168144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923c034d70 0x7f923c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 monclient: found mon.noname-a 2023-11-02T15:34:04.378 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f2f9c5eb640 10 monclient: get_monmap_and_config 2023-11-02T15:34:04.378 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f2f9c5eb640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.378 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.379 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.379 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f2f9c5eb640 10 monclient: monmap: 2023-11-02T15:34:04.379 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.379 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:04.255629+0000 2023-11-02T15:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:04.255629+0000 2023-11-02T15:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:04.381 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:04.381 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:04.381 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:04.381 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:04.381 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.382 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f9241d9b640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009400 con 0x7f923c0c3c80 2023-11-02T15:34:04.382 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.253+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416d43f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416d43f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416d43f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f2f9c5eb640 1 Processor -- start 2023-11-02T15:34:04.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f2f9c5eb640 1 -- start start 2023-11-02T15:34:04.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f2f9c5eb640 10 monclient: init 2023-11-02T15:34:04.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416dc40640 10 monclient(hunting): get_auth_request con 0x7f41680a84b0 auth_method 0 2023-11-02T15:34:04.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416dc40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416dc40640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.254+0000 7f416dc40640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7ff55cf41640 1 --2- 172.21.15.174:0/3025194639 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec35bc4446e2fc15 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.388 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient: _add_conns ranks=[0,3,2,4,1] 2023-11-02T15:34:04.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f940b4570 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:04.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2f940b0ed0 0x7f2f94144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-a con 0x7f2f940b0ed0 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:04.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f9413e520 0x7f2f9413e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-b con 0x7f2f9413e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f940a9780 con 0x7f2f940b4570 2023-11-02T15:34:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.255+0000 7f2f9c5eb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f940b4950 con 0x7f2f940b0ed0 2023-11-02T15:34:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f2f9c5eb640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f2f9c5eb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940aa1d0 con 0x7f2f9413e520 2023-11-02T15:34:04.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f2f9c5eb640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f2f9c5eb640 10 monclient(hunting): authenticate will time out at 1371.598733s 2023-11-02T15:34:04.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38884/0 (socket says 172.21.15.174:38884) 2023-11-02T15:34:04.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 1 -- 172.21.15.174:0/3173023240 learned_addr learned my addr 172.21.15.174:0/3173023240 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 10 monclient(hunting): get_auth_request con 0x7f67fc0b3220 auth_method 0 2023-11-02T15:34:04.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f67fbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f416d43f640 10 monclient(hunting): handle_auth_done global_id 4281 payload 274 2023-11-02T15:34:04.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f416d43f640 1 -- 172.21.15.174:0/589037048 >> v2:172.21.15.175:3300/0 conn(0x7f41680a84b0 msgr2=0x7f4168144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f416d43f640 1 --2- 172.21.15.174:0/589037048 >> v2:172.21.15.175:3300/0 conn(0x7f41680a84b0 0x7f4168144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.256+0000 7f416d43f640 1 -- 172.21.15.174:0/589037048 >> v1:172.21.15.175:6789/0 conn(0x7f41680b0ec0 legacy=0x7f41680aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.257+0000 7f416d43f640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.257+0000 7f416d43f640 1 monclient: found mon.noname-a 2023-11-02T15:34:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.257+0000 7f416d43f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.257+0000 7f416d43f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.257+0000 7f416d43f640 1 -- 172.21.15.174:0/589037048 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680b0680 con 0x7f41680a8890 2023-11-02T15:34:04.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.258+0000 7f67fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.258+0000 7f67fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.258+0000 7f67fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.258+0000 7f6801704640 1 --2- 172.21.15.174:0/3173023240 >> v2:172.21.15.175:3300/0 conn(0x7f67fc13ef10 0x7f67fc144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.259+0000 7f9241d9b640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 0x7f923c0c0cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f92340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.259+0000 7f9232ffd640 10 client.?.objecter ms_handle_connect 0x7f923c0c3c80 2023-11-02T15:34:04.398 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.259+0000 7f9232ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.398 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.259+0000 7f57577fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f575813c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.398 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f6801704640 10 monclient(hunting): get_auth_request con 0x7f67fc13ef10 auth_method 0 2023-11-02T15:34:04.399 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f6801704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.399 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f6801704640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.399 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f6801704640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f575813c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57890/0 (socket says 172.21.15.174:57890) 2023-11-02T15:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 1 -- 172.21.15.174:0/2101142167 learned_addr learned my addr 172.21.15.174:0/2101142167 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 10 monclient(hunting): get_auth_request con 0x7f57580a9eb0 auth_method 0 2023-11-02T15:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.260+0000 7f57577fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f57577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f416d43f640 1 --2- 172.21.15.174:0/589037048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680a8890 0x7f41680b7240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41680b2c70 tx=0x7f415802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bce189e602d8dac server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 10 monclient(hunting): handle_auth_done global_id 4253 payload 274 2023-11-02T15:34:04.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 1 -- 172.21.15.174:0/3173023240 >> v2:172.21.15.175:3300/0 conn(0x7f67fc13ef10 msgr2=0x7f67fc144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 1 --2- 172.21.15.174:0/3173023240 >> v2:172.21.15.175:3300/0 conn(0x7f67fc13ef10 0x7f67fc144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 1 -- 172.21.15.174:0/3173023240 >> v1:172.21.15.175:6789/0 conn(0x7f67fc0b3b70 legacy=0x7f67fc13e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.261+0000 7f67fbfff640 1 -- 172.21.15.174:0/3173023240 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0a8ce0 con 0x7f67fc0b3220 2023-11-02T15:34:04.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.264+0000 7f2f99b86640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2f940b0ed0 0x7f2f94144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 10 monclient(hunting): handle_auth_done global_id 4294 payload 824 2023-11-02T15:34:04.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 -- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57580a9470 msgr2=0x7f57581369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57580a9470 0x7f57581369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 -- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57580aa290 msgr2=0x7f57581c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57580aa290 0x7f57581c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.265+0000 7f57577fe640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575000d050 con 0x7f57580a9eb0 2023-11-02T15:34:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2f940b0ed0 0x7f2f94144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38820/0 (socket says 172.21.15.174:38820) 2023-11-02T15:34:04.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 1 -- 172.21.15.174:0/1705718339 learned_addr learned my addr 172.21.15.174:0/1705718339 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 10 monclient(hunting): get_auth_request con 0x7f2f940b0ed0 auth_method 0 2023-11-02T15:34:04.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f2f99b86640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 1 -- 172.21.15.174:0/3025194639 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0317c0 con 0x7ff5580abe50 2023-11-02T15:34:04.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7f954daa6640 1 --2- 172.21.15.174:0/3977946388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc598fb506709bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: dump: 2023-11-02T15:34:04.411 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.411 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.411 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.411 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.411 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.412 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.412 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.412 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.412 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.412 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff55e9a4640 5 monclient: authenticate success, global_id 4250 2023-11-02T15:34:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff55e9a4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 1 -- 172.21.15.174:0/3025194639 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c031920 con 0x7ff5580abe50 2023-11-02T15:34:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff556ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff55e9a4640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.266+0000 7ff55e9a4640 4 set_mon_vals no callback set 2023-11-02T15:34:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff556ffd640 1 -- 172.21.15.174:0/3025194639 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0317c0 con 0x7ff5580abe50 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff556ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff556ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff556ffd640 10 monclient: dump: 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.415 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.416 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.416 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.416 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.416 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.416 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.417 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.417 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7f2f9ab88640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f9413e520 0x7f2f9413e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 10 monclient: shutdown 2023-11-02T15:34:04.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3025194639 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 msgr2=0x7ff5580ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3025194639 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02b860 comp rx=0 tx=0).stop 2023-11-02T15:34:04.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3025194639 shutdown_connections 2023-11-02T15:34:04.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3025194639 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5580abe50 0x7ff5580ac2c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3025194639 >> v2:172.21.15.175:3300/0 conn(0x7ff55806a940 0x7ff5580aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3025194639 >> 172.21.15.174:0/3025194639 conn(0x7ff558069e50 msgr2=0x7ff55806a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3025194639 shutdown_connections 2023-11-02T15:34:04.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3025194639 wait complete. 2023-11-02T15:34:04.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.267+0000 7f67fbfff640 1 --2- 172.21.15.174:0/3173023240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009870 tx=0x7f67e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c56b1e7ee4446b server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7ff55e9a4640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7ff55e9a4640 10 monclient: monmap: 2023-11-02T15:34:04.422 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.423 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.423 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.423 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.423 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.423 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.424 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.424 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.424 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.424 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234003610 con 0x7f923c0c3c80 2023-11-02T15:34:04.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:04.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: dump: 2023-11-02T15:34:04.426 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.426 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.426 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.426 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.426 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234003770 con 0x7f923c0c3c80 2023-11-02T15:34:04.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7ff55e9a4640 1 librados: starting msgr at 2023-11-02T15:34:04.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7ff55e9a4640 1 librados: starting objecter 2023-11-02T15:34:04.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234003b20 con 0x7f923c0c3c80 2023-11-02T15:34:04.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:04.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9232ffd640 10 monclient: dump: 2023-11-02T15:34:04.431 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.431 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.431 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.432 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.432 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.432 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.432 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.432 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 5 monclient: authenticate success, global_id 4278 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 10 monclient: _renew_subs 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c0c0090 con 0x7f923c0c3c80 2023-11-02T15:34:04.434 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7ff55e9a4640 1 Processor -- start 2023-11-02T15:34:04.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f9240d99640 4 set_mon_vals no callback set 2023-11-02T15:34:04.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 10 monclient: _renew_subs 2023-11-02T15:34:04.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c0cab30 con 0x7f923c0c3c80 2023-11-02T15:34:04.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.268+0000 7f924359e640 1 librados: init done 2023-11-02T15:34:04.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 -- start start 2023-11-02T15:34:04.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 librados: setting wanted keys 2023-11-02T15:34:04.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 librados: calling monclient init 2023-11-02T15:34:04.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient: init 2023-11-02T15:34:04.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f2f9ab88640 10 monclient(hunting): get_auth_request con 0x7f2f9413e520 auth_method 0 2023-11-02T15:34:04.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f2f9ab88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f2f9ab88640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f2f9ab88640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:04.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff55806a940 0x7ff5580b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-c con 0x7ff55806a940 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5580a94c0 0x7ff55813d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-a con 0x7ff5580a94c0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 0x7ff558129150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): picked mon.noname-b con 0x7ff5580abe50 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55813d650 con 0x7ff5580a94c0 2023-11-02T15:34:04.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55813dea0 con 0x7ff5580abe50 2023-11-02T15:34:04.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 10 client.4278.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 15 client.4278.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 20 client.4278.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f923c0c4630 con 0x7f923c0c3c80 2023-11-02T15:34:04.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55806dca0 con 0x7ff55806a940 2023-11-02T15:34:04.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.269+0000 7ff55e9a4640 10 monclient(hunting): authenticate will time out at 1371.612510s 2023-11-02T15:34:04.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 1 -- 172.21.15.174:0/3061828950 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4038600 con 0x7f85dc0ac560 2023-11-02T15:34:04.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: dump: 2023-11-02T15:34:04.445 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.445 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.446 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.446 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.446 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.446 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.446 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 1 -- 172.21.15.174:0/3061828950 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d4032070 con 0x7f85dc0ac560 2023-11-02T15:34:04.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 1 -- 172.21.15.174:0/3061828950 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40389c0 con 0x7f85dc0ac560 2023-11-02T15:34:04.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85e21dd640 5 monclient: authenticate success, global_id 4247 2023-11-02T15:34:04.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85e21dd640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85e21dd640 4 set_mon_vals no callback set 2023-11-02T15:34:04.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f85da7fc640 10 monclient: dump: 2023-11-02T15:34:04.449 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.450 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.450 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.450 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.450 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.450 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.451 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.451 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.451 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.451 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.270+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 10 monclient(hunting): handle_auth_done global_id 4284 payload 274 2023-11-02T15:34:04.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 1 -- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f9413e520 msgr2=0x7f2f9413e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:04.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f9413e520 0x7f2f9413e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 1 -- 172.21.15.174:0/1705718339 >> v1:172.21.15.174:6789/0 conn(0x7f2f940b4570 legacy=0x7f2f940a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 1 monclient: found mon.noname-a 2023-11-02T15:34:04.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f2f99b86640 1 -- 172.21.15.174:0/1705718339 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940aa4b0 con 0x7f2f940b0ed0 2023-11-02T15:34:04.455 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 10 monclient: shutdown 2023-11-02T15:34:04.455 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.455 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 -- 172.21.15.174:0/3061828950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 msgr2=0x7f85dc149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 --2- 172.21.15.174:0/3061828950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85d400d3a0 tx=0x7f85d402e310 comp rx=0 tx=0).stop 2023-11-02T15:34:04.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 -- 172.21.15.174:0/3061828950 shutdown_connections 2023-11-02T15:34:04.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 --2- 172.21.15.174:0/3061828950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc149120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 -- 172.21.15.174:0/3061828950 >> 172.21.15.174:0/3061828950 conn(0x7f85dc069e50 msgr2=0x7f85dc06a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f57577fe640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f575813c8e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e870 tx=0x7f574c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f5754ff9640 10 client.?.objecter ms_handle_connect 0x7f57580a9eb0 2023-11-02T15:34:04.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f5754ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.458 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 -- 172.21.15.174:0/3061828950 shutdown_connections 2023-11-02T15:34:04.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.271+0000 7f85e21dd640 1 -- 172.21.15.174:0/3061828950 wait complete. 2023-11-02T15:34:04.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 10 monclient: monmap: 2023-11-02T15:34:04.459 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.460 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.460 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.461 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.461 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.461 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.461 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.461 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.462 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.462 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 librados: starting msgr at 2023-11-02T15:34:04.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 librados: starting objecter 2023-11-02T15:34:04.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 Processor -- start 2023-11-02T15:34:04.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 -- start start 2023-11-02T15:34:04.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 librados: setting wanted keys 2023-11-02T15:34:04.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 1 librados: calling monclient init 2023-11-02T15:34:04.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.272+0000 7f85e21dd640 10 monclient: init 2023-11-02T15:34:04.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:34:04.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85dc0a9730 0x7f85dc0ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-a con 0x7f85dc0a9730 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-c con 0x7f85dc0ac560 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0aeef0 0x7f85dc130aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): picked mon.noname-b con 0x7f85dc0aeef0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b11d0 con 0x7f85dc0a9730 2023-11-02T15:34:04.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b13b0 con 0x7f85dc0aeef0 2023-11-02T15:34:04.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc132670 con 0x7f85dc0ac560 2023-11-02T15:34:04.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.273+0000 7f85e21dd640 10 monclient(hunting): authenticate will time out at 1371.616039s 2023-11-02T15:34:04.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.274+0000 7f2f99b86640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f940b0ed0 0x7f2f94144d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009880 tx=0x7f2f8402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42ca3b4ccb06bdc server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0094f0 con 0x7f57580a9eb0 2023-11-02T15:34:04.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: dump: 2023-11-02T15:34:04.468 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.469 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.469 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.471 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c00c940 con 0x7f57580a9eb0 2023-11-02T15:34:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 5 monclient: authenticate success, global_id 4294 2023-11-02T15:34:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 10 monclient: _renew_subs 2023-11-02T15:34:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57581d1ca0 con 0x7f57580a9eb0 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c00ccf0 con 0x7f57580a9eb0 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f5754ff9640 10 monclient: dump: 2023-11-02T15:34:04.473 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.475 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.475 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.475 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.475 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 10 client.4294.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 10 monclient: _renew_subs 2023-11-02T15:34:04.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57581d2200 con 0x7f57580a9eb0 2023-11-02T15:34:04.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f57567fc640 4 set_mon_vals no callback set 2023-11-02T15:34:04.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7f575d6cc640 1 librados: init done 2023-11-02T15:34:04.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.277+0000 7ff55cf41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 0x7ff558129150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 10 client.4294.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 15 client.4294.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 20 client.4294.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 10 client.4294.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57581c56d0 con 0x7f57580a9eb0 2023-11-02T15:34:04.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 0x7ff558129150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57900/0 (socket says 172.21.15.174:57900) 2023-11-02T15:34:04.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 1 -- 172.21.15.174:0/3140965423 learned_addr learned my addr 172.21.15.174:0/3140965423 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 10 monclient(hunting): get_auth_request con 0x7ff5580abe50 auth_method 0 2023-11-02T15:34:04.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.278+0000 7ff55cf41640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.280+0000 7f67fb7fe640 1 -- 172.21.15.174:0/3173023240 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8031640 con 0x7f67fc0b3220 2023-11-02T15:34:04.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.280+0000 7f67fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f67fb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f67fb7fe640 10 monclient: dump: 2023-11-02T15:34:04.483 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.483 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.484 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.484 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.484 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.485 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.485 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.485 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.485 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.485 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f67fb7fe640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f67fb7fe640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f67fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f6801f05640 5 monclient: authenticate success, global_id 4253 2023-11-02T15:34:04.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.281+0000 7f6801f05640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:04.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 1 -- 172.21.15.174:0/3173023240 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e80317a0 con 0x7f67fc0b3220 2023-11-02T15:34:04.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 1 -- 172.21.15.174:0/3173023240 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8031b30 con 0x7f67fc0b3220 2023-11-02T15:34:04.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f67fb7fe640 10 monclient: dump: 2023-11-02T15:34:04.488 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.488 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.488 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.489 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.490 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f6801f05640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.282+0000 7f6801f05640 4 set_mon_vals no callback set 2023-11-02T15:34:04.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 10 monclient: shutdown 2023-11-02T15:34:04.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 -- 172.21.15.174:0/3173023240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 msgr2=0x7f67fc0a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 --2- 172.21.15.174:0/3173023240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009870 tx=0x7f67e802b8f0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 -- 172.21.15.174:0/3173023240 shutdown_connections 2023-11-02T15:34:04.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 --2- 172.21.15.174:0/3173023240 >> v2:172.21.15.175:3300/0 conn(0x7f67fc13ef10 0x7f67fc144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 --2- 172.21.15.174:0/3173023240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc0b3220 0x7f67fc0a86c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 -- 172.21.15.174:0/3173023240 >> 172.21.15.174:0/3173023240 conn(0x7f67fc065960 msgr2=0x7f67fc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 -- 172.21.15.174:0/3173023240 shutdown_connections 2023-11-02T15:34:04.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.283+0000 7f6801f05640 1 -- 172.21.15.174:0/3173023240 wait complete. 2023-11-02T15:34:04.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 10 monclient: monmap: 2023-11-02T15:34:04.493 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.493 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.493 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.493 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.493 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.494 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.494 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.494 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.494 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.494 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 librados: starting msgr at 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 librados: starting objecter 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 Processor -- start 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 -- start start 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 librados: setting wanted keys 2023-11-02T15:34:04.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 1 librados: calling monclient init 2023-11-02T15:34:04.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7f6801f05640 10 monclient: init 2023-11-02T15:34:04.497 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.284+0000 7ff557fff640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff55806a940 0x7ff5580b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T15:34:04.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67fc0b3b70 0x7f67fc144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-a con 0x7f67fc0b3b70 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67fc13ef10 0x7f67fc141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-b con 0x7f67fc13ef10 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 0x7f67fc1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): picked mon.noname-c con 0x7f67fc142410 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc144bc0 con 0x7f67fc0b3b70 2023-11-02T15:34:04.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1448a0 con 0x7f67fc13ef10 2023-11-02T15:34:04.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc142de0 con 0x7f67fc142410 2023-11-02T15:34:04.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.285+0000 7f6801f05640 10 monclient(hunting): authenticate will time out at 1371.628119s 2023-11-02T15:34:04.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.286+0000 7ff557fff640 10 monclient(hunting): get_auth_request con 0x7ff55806a940 auth_method 0 2023-11-02T15:34:04.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.286+0000 7ff557fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.286+0000 7ff557fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.286+0000 7ff557fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 1 -- 172.21.15.174:0/3977946388 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544031870 con 0x7f95480a9730 2023-11-02T15:34:04.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: dump: 2023-11-02T15:34:04.503 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.504 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.505 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.505 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.505 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 1 -- 172.21.15.174:0/3977946388 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440319d0 con 0x7f95480a9730 2023-11-02T15:34:04.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 1 -- 172.21.15.174:0/3977946388 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544031d60 con 0x7f95480a9730 2023-11-02T15:34:04.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954caa4640 10 monclient: dump: 2023-11-02T15:34:04.507 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.507 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.509 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.509 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.509 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.509 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.510 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.510 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.510 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.510 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f85daffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff55cf41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954fd0a640 5 monclient: authenticate success, global_id 4244 2023-11-02T15:34:04.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954fd0a640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.287+0000 7f954fd0a640 4 set_mon_vals no callback set 2023-11-02T15:34:04.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 10 monclient: shutdown 2023-11-02T15:34:04.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 -- 172.21.15.174:0/3977946388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 msgr2=0x7f95480aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 --2- 172.21.15.174:0/3977946388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954402b910 comp rx=0 tx=0).stop 2023-11-02T15:34:04.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 -- 172.21.15.174:0/3977946388 shutdown_connections 2023-11-02T15:34:04.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 --2- 172.21.15.174:0/3977946388 >> v2:172.21.15.174:3300/0 conn(0x7f954813b000 0x7f95480ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 --2- 172.21.15.174:0/3977946388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95480a9730 0x7f95480aeef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 -- 172.21.15.174:0/3977946388 >> 172.21.15.174:0/3977946388 conn(0x7f9548069e50 msgr2=0x7f954806a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff557fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff557fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7ff557fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 -- 172.21.15.174:0/3977946388 shutdown_connections 2023-11-02T15:34:04.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.288+0000 7f954fd0a640 1 -- 172.21.15.174:0/3977946388 wait complete. 2023-11-02T15:34:04.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.289+0000 7f954fd0a640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.289+0000 7f954fd0a640 10 monclient: monmap: 2023-11-02T15:34:04.518 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.518 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.518 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.518 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.518 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.519 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.519 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.519 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.519 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.519 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.289+0000 7f954fd0a640 1 librados: starting msgr at 2023-11-02T15:34:04.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.289+0000 7f954fd0a640 1 librados: starting objecter 2023-11-02T15:34:04.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 Processor -- start 2023-11-02T15:34:04.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 -- start start 2023-11-02T15:34:04.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 librados: setting wanted keys 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 librados: calling monclient init 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient: init 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95480a9730 0x7f954813dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-a con 0x7f95480a9730 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 0x7f9548139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-c con 0x7f954813abc0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954813b000 0x7f95481d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): picked mon.noname-b con 0x7f954813b000 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95481d5830 con 0x7f95480a9730 2023-11-02T15:34:04.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954813b6f0 con 0x7f954813b000 2023-11-02T15:34:04.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95481d2170 con 0x7f954813abc0 2023-11-02T15:34:04.524 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.524 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7f954fd0a640 10 monclient(hunting): authenticate will time out at 1371.633382s 2023-11-02T15:34:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7ff55cf41640 10 monclient(hunting): handle_auth_done global_id 4303 payload 824 2023-11-02T15:34:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7ff55cf41640 1 -- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff55806a940 msgr2=0x7ff5580b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.290+0000 7ff55cf41640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff55806a940 0x7ff5580b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 1 -- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5580a94c0 msgr2=0x7ff55813d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5580a94c0 0x7ff55813d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.291+0000 7ff55cf41640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0094d0 con 0x7ff5580abe50 2023-11-02T15:34:04.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.292+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0aeef0 0x7f85dc130aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0aeef0 0x7f85dc130aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57908/0 (socket says 172.21.15.174:57908) 2023-11-02T15:34:04.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 1 -- 172.21.15.174:0/2632788559 learned_addr learned my addr 172.21.15.174:0/2632788559 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.529 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 10 monclient(hunting): get_auth_request con 0x7f85dc0aeef0 auth_method 0 2023-11-02T15:34:04.529 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.529 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.529 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.293+0000 7f85dbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.529 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.294+0000 7f85daffd640 10 monclient(hunting): get_auth_request con 0x7f85dc0ac560 auth_method 0 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.294+0000 7f85daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.294+0000 7f85daffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.294+0000 7f85daffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.296+0000 7f85daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.296+0000 7f85daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.530 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.296+0000 7f85daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.297+0000 7f6801704640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 0x7f67fc1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f954d2a5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 0x7f9548139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 0x7f67fc1428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38934/0 (socket says 172.21.15.174:38934) 2023-11-02T15:34:04.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 1 -- 172.21.15.174:0/699461796 learned_addr learned my addr 172.21.15.174:0/699461796 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 10 monclient(hunting): get_auth_request con 0x7f67fc142410 auth_method 0 2023-11-02T15:34:04.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.298+0000 7f6801704640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.299+0000 7f6801704640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.299+0000 7f6801704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.299+0000 7f6801704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 0x7f9548139e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38940/0 (socket says 172.21.15.174:38940) 2023-11-02T15:34:04.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 1 -- 172.21.15.174:0/336913061 learned_addr learned my addr 172.21.15.174:0/336913061 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 10 monclient(hunting): get_auth_request con 0x7f954813abc0 auth_method 0 2023-11-02T15:34:04.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.300+0000 7f954d2a5640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2023-11-02T15:34:04.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67fc13ef10 msgr2=0x7f67fc141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67fc13ef10 0x7f67fc141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67fc0b3b70 msgr2=0x7f67fc144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67fc0b3b70 0x7f67fc144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 monclient: found mon.noname-c 2023-11-02T15:34:04.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.302+0000 7f6801704640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80094d0 con 0x7f67fc142410 2023-11-02T15:34:04.538 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.304+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f923402f050 con 0x7f923c0c3c80 2023-11-02T15:34:04.538 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.304+0000 7f9232ffd640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f922406bdc0 0x7f922406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.539 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.304+0000 7f924259c640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f922406bdc0 0x7f922406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.539 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.304+0000 7f924259c640 10 monclient: get_auth_request con 0x7f922406bdc0 auth_method 0 2023-11-02T15:34:04.539 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.305+0000 7f9232ffd640 10 client.4278.objecter ms_handle_connect 0x7f922406bdc0 2023-11-02T15:34:04.539 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.305+0000 7f924259c640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f922406bdc0 0x7f922406e280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f924404f940 tx=0x7f9244073200 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.540 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.306+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f574c013470 con 0x7f57580a9eb0 2023-11-02T15:34:04.540 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.306+0000 7f5754ff9640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f574406bdf0 0x7f574406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.306+0000 7f5757fff640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f574406bdf0 0x7f574406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.306+0000 7f5757fff640 10 monclient: get_auth_request con 0x7f574406bdf0 auth_method 0 2023-11-02T15:34:04.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f5757fff640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f574406bdf0 0x7f574406e2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5750038000 tx=0x7f5750030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f5754ff9640 10 client.4294.objecter ms_handle_connect 0x7f574406bdf0 2023-11-02T15:34:04.542 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 10 monclient(hunting): handle_auth_done global_id 4262 payload 824 2023-11-02T15:34:04.542 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 -- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0aeef0 msgr2=0x7f85dc130aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.542 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc0aeef0 0x7f85dc130aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 -- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85dc0a9730 msgr2=0x7f85dc0ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85dc0a9730 0x7f85dc0ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 monclient: found mon.noname-c 2023-11-02T15:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.307+0000 7f85daffd640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d400d050 con 0x7f85dc0ac560 2023-11-02T15:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f954d2a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.545 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f954d2a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.545 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f954d2a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f923407a560 con 0x7f923c0c3c80 2023-11-02T15:34:04.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 10 client.4278.objecter ms_dispatch 0x7f9244000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 3 client.4278.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 3 client.4278.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 10 client.4278.objecter checking command 1 2023-11-02T15:34:04.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f92240721a0 0x7f92240745c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter _get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 15 client.4278.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter _get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 15 client.4278.objecter _session_command_op_assign 2 1 2023-11-02T15:34:04.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 4 2023-11-02T15:34:04.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 10 client.4278.objecter _send_command 1 2023-11-02T15:34:04.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9224074c70 con 0x7f92240721a0 2023-11-02T15:34:04.549 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 20 client.4278.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.549 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92340761a0 con 0x7f923c0c3c80 2023-11-02T15:34:04.549 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f9232ffd640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.550 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 1 -- 172.21.15.174:0/589037048 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158031750 con 0x7f41680a8890 2023-11-02T15:34:04.550 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.551 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:04.551 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: dump: 2023-11-02T15:34:04.551 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.551 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.551 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.553 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 1 -- 172.21.15.174:0/589037048 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41580318b0 con 0x7f41680a8890 2023-11-02T15:34:04.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416ec42640 5 monclient: authenticate success, global_id 4281 2023-11-02T15:34:04.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416ec42640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:04.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416ec42640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.554 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 1 -- 172.21.15.174:0/589037048 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158031c40 con 0x7f41680a8890 2023-11-02T15:34:04.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416ec42640 4 set_mon_vals no callback set 2023-11-02T15:34:04.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:04.555 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.308+0000 7f416cc3e640 10 monclient: dump: 2023-11-02T15:34:04.555 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.556 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.557 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f9242d9d640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f92240721a0 0x7f92240745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f9242d9d640 10 monclient: get_auth_request con 0x7f92240721a0 auth_method 0 2023-11-02T15:34:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f416ec42640 10 monclient: shutdown 2023-11-02T15:34:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f416ec42640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f416ec42640 1 -- 172.21.15.174:0/589037048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680a8890 msgr2=0x7f41680b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.309+0000 7f416ec42640 1 --2- 172.21.15.174:0/589037048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680a8890 0x7f41680b7240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41680b2c70 tx=0x7f415802b7f0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 -- 172.21.15.174:0/589037048 shutdown_connections 2023-11-02T15:34:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 --2- 172.21.15.174:0/589037048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680a8890 0x7f41680b7240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 --2- 172.21.15.174:0/589037048 >> v2:172.21.15.175:3300/0 conn(0x7f41680a84b0 0x7f4168144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 -- 172.21.15.174:0/589037048 >> 172.21.15.174:0/589037048 conn(0x7f4168065b40 msgr2=0x7f4168065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 -- 172.21.15.174:0/589037048 shutdown_connections 2023-11-02T15:34:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.310+0000 7f416ec42640 1 -- 172.21.15.174:0/589037048 wait complete. 2023-11-02T15:34:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 10 monclient: monmap: 2023-11-02T15:34:04.562 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.562 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.562 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.562 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.562 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.563 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.563 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.563 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.563 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.563 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.564 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f574c058740 con 0x7f57580a9eb0 2023-11-02T15:34:04.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 10 client.4294.objecter ms_dispatch 0x7f5758000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 3 client.4294.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 3 client.4294.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 10 client.4294.objecter checking command 1 2023-11-02T15:34:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7f57440721d0 0x7f57440745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter _get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 15 client.4294.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter _get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 15 client.4294.objecter _session_command_op_assign 0 1 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 4 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 10 client.4294.objecter _send_command 1 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5744074ca0 con 0x7f57440721d0 2023-11-02T15:34:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 20 client.4294.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 librados: starting msgr at 2023-11-02T15:34:04.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 librados: starting objecter 2023-11-02T15:34:04.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 Processor -- start 2023-11-02T15:34:04.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 -- start start 2023-11-02T15:34:04.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 librados: setting wanted keys 2023-11-02T15:34:04.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 1 librados: calling monclient init 2023-11-02T15:34:04.570 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f416ec42640 10 monclient: init 2023-11-02T15:34:04.570 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f574c054380 con 0x7f57580a9eb0 2023-11-02T15:34:04.570 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.311+0000 7f5754ff9640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.571 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.571 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:04.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 0x7f41680ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-b con 0x7f41680a84b0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680b0ec0 0x7f41681d6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-a con 0x7f41680b0ec0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41681d6db0 0x7f41680ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): picked mon.noname-c con 0x7f41681d6db0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681d6130 con 0x7f41680b0ec0 2023-11-02T15:34:04.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681d6350 con 0x7f41680a84b0 2023-11-02T15:34:04.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680ab760 con 0x7f41681d6db0 2023-11-02T15:34:04.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f416ec42640 10 monclient(hunting): authenticate will time out at 1371.655038s 2023-11-02T15:34:04.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f575cecb640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7f57440721d0 0x7f57440745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7ff55cf41640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 0x7ff558129150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff54c02b8a0 tx=0x7ff54c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7ff554ff9640 10 client.?.objecter ms_handle_connect 0x7ff5580abe50 2023-11-02T15:34:04.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7ff554ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.312+0000 7f575cecb640 10 monclient: get_auth_request con 0x7f57440721d0 auth_method 0 2023-11-02T15:34:04.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 10 monclient(hunting): handle_auth_done global_id 4268 payload 824 2023-11-02T15:34:04.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 -- 172.21.15.174:0/336913061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954813b000 msgr2=0x7f95481d5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f954813b000 0x7f95481d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 -- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95480a9730 msgr2=0x7f954813dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95480a9730 0x7f954813dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 monclient: found mon.noname-c 2023-11-02T15:34:04.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.313+0000 7f954d2a5640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440094d0 con 0x7f954813abc0 2023-11-02T15:34:04.579 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.314+0000 7f9242d9d640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f92240721a0 0x7f92240745c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.579 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.314+0000 7f9232ffd640 10 client.4278.objecter ms_handle_connect 0x7f92240721a0 2023-11-02T15:34:04.579 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.315+0000 7f416dc40640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 0x7f41680ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.316+0000 7f2f99385640 1 -- 172.21.15.174:0/1705718339 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8402e070 con 0x7f2f940b0ed0 2023-11-02T15:34:04.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.316+0000 7f2f99385640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.316+0000 7f2f99385640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:04.581 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.316+0000 7f2f99385640 10 monclient: dump: 2023-11-02T15:34:04.581 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.581 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.581 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.581 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.582 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.582 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.582 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.582 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.582 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.583 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 1 -- 172.21.15.174:0/1705718339 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84004030 con 0x7f2f940b0ed0 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 1 -- 172.21.15.174:0/1705718339 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84004420 con 0x7f2f940b0ed0 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f99385640 10 monclient: dump: 2023-11-02T15:34:04.584 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.585 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.585 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.586 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.586 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.586 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.586 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.586 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.587 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.587 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 5 monclient: authenticate success, global_id 4284 2023-11-02T15:34:04.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:04.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 4 set_mon_vals no callback set 2023-11-02T15:34:04.588 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 0x7f41680ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57944/0 (socket says 172.21.15.174:57944) 2023-11-02T15:34:04.588 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f575cecb640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7f57440721d0 0x7f57440745f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 1 -- 172.21.15.174:0/4139576698 learned_addr learned my addr 172.21.15.174:0/4139576698 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 10 monclient(hunting): get_auth_request con 0x7f41680a84b0 auth_method 0 2023-11-02T15:34:04.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f416dc40640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f6801704640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 0x7f67fc1428a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f67f4007ec0 tx=0x7f67f400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f67f97fa640 10 client.?.objecter ms_handle_connect 0x7f67fc142410 2023-11-02T15:34:04.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f67f97fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f5754ff9640 10 client.4294.objecter ms_handle_connect 0x7f57440721d0 2023-11-02T15:34:04.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 10 monclient: shutdown 2023-11-02T15:34:04.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f940b0ed0 msgr2=0x7f2f94144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.317+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f940b0ed0 0x7f2f94144d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009880 tx=0x7f2f8402b840 comp rx=0 tx=0).stop 2023-11-02T15:34:04.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/1705718339 shutdown_connections 2023-11-02T15:34:04.593 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f9413e520 0x7f2f9413e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.593 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/1705718339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f940b0ed0 0x7f2f94144d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.593 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/1705718339 >> 172.21.15.174:0/1705718339 conn(0x7f2f94065d50 msgr2=0x7f2f94066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.593 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/1705718339 shutdown_connections 2023-11-02T15:34:04.593 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/1705718339 wait complete. 2023-11-02T15:34:04.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.318+0000 7f416dc40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 10 monclient: build_initial_monmap 2023-11-02T15:34:04.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f85daffd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc0b0c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d000b3e0 tx=0x7f85d000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f85c3fff640 10 client.?.objecter ms_handle_connect 0x7f85dc0ac560 2023-11-02T15:34:04.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f85c3fff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.597 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 10 monclient: monmap: 2023-11-02T15:34:04.598 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:04.598 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:04.598 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:04.598 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:04.599 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:04.599 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.599 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:04.599 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:04.599 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:04.600 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 librados: starting msgr at 2023-11-02T15:34:04.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 librados: starting objecter 2023-11-02T15:34:04.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 Processor -- start 2023-11-02T15:34:04.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 -- start start 2023-11-02T15:34:04.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 librados: setting wanted keys 2023-11-02T15:34:04.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 1 librados: calling monclient init 2023-11-02T15:34:04.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.319+0000 7f2f9c5eb640 10 monclient: init 2023-11-02T15:34:04.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:04.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:04.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f940b0ed0 0x7f2f940aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-c con 0x7f2f940b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:04.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f940b4570 0x7f2f940ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-b con 0x7f2f940b4570 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:04.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 0x7f2f941d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): picked mon.noname-a con 0x7f2f9413e520 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940ab5c0 con 0x7f2f9413e520 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f941d6730 con 0x7f2f940b4570 2023-11-02T15:34:04.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:04.604 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f94141f80 con 0x7f2f940b0ed0 2023-11-02T15:34:04.604 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:04.604 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.320+0000 7f2f9c5eb640 10 monclient(hunting): authenticate will time out at 1371.663356s 2023-11-02T15:34:04.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.321+0000 7f954d2a5640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 0x7f9548139e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f954000c7a0 tx=0x7f954000cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.321+0000 7f953e7fc640 10 client.?.objecter ms_handle_connect 0x7f954813abc0 2023-11-02T15:34:04.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.321+0000 7f953e7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7f2f9ab88640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 0x7f2f941d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c003810 con 0x7ff5580abe50 2023-11-02T15:34:04.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.607 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.607 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 10 monclient: dump: 2023-11-02T15:34:04.607 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.607 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.607 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.608 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.609 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.609 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7f2f99b86640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f940b4570 0x7f2f940ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.612 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.612 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.612 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff554ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.612 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 5 monclient: authenticate success, global_id 4303 2023-11-02T15:34:04.612 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 10 monclient: _renew_subs 2023-11-02T15:34:04.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff558129690 con 0x7ff5580abe50 2023-11-02T15:34:04.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 10 client.4303.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 10 monclient: _renew_subs 2023-11-02T15:34:04.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55813d030 con 0x7ff5580abe50 2023-11-02T15:34:04.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.322+0000 7ff55e9a4640 1 librados: init done 2023-11-02T15:34:04.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c040070 con 0x7ff5580abe50 2023-11-02T15:34:04.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c003b90 con 0x7ff5580abe50 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff554ff9640 10 monclient: dump: 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.615 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.616 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.616 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.616 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.616 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.616 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.617 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.617 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.617 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff5567fc640 4 set_mon_vals no callback set 2023-11-02T15:34:04.617 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 10 client.4303.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.617 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 15 client.4303.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 20 client.4303.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 10 client.4303.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.323+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5380028a0 con 0x7ff5580abe50 2023-11-02T15:34:04.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 0x7f2f941d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38870/0 (socket says 172.21.15.174:38870) 2023-11-02T15:34:04.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 1 -- 172.21.15.174:0/3822503894 learned_addr learned my addr 172.21.15.174:0/3822503894 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:04.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): get_auth_request con 0x7f2f9413e520 auth_method 0 2023-11-02T15:34:04.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.324+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 10 monclient(hunting): handle_auth_done global_id 4309 payload 824 2023-11-02T15:34:04.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 -- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41681d6db0 msgr2=0x7f41680ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41681d6db0 0x7f41680ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 -- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680b0ec0 msgr2=0x7f41681d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41680b0ec0 0x7f41681d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 monclient: found mon.noname-b 2023-11-02T15:34:04.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f416dc40640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580094d0 con 0x7f41680a84b0 2023-11-02T15:34:04.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f2f99b86640 10 monclient(hunting): get_auth_request con 0x7f2f940b4570 auth_method 0 2023-11-02T15:34:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f2f99b86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f2f99b86640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.325+0000 7f2f99b86640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.327+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.327+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:04.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.327+0000 7f2f99b86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:04.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 10 monclient(hunting): handle_auth_done global_id 4290 payload 824 2023-11-02T15:34:04.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 -- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f940b0ed0 msgr2=0x7f2f940aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f940b0ed0 0x7f2f940aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 -- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f940b4570 msgr2=0x7f2f940ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f940b4570 0x7f2f940ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 10 monclient: _finish_hunting 0 2023-11-02T15:34:04.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 monclient: found mon.noname-a 2023-11-02T15:34:04.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:04.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f2f9ab88640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84009460 con 0x7f2f9413e520 2023-11-02T15:34:04.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540003d80 con 0x7f954813abc0 2023-11-02T15:34:04.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:04.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: dump: 2023-11-02T15:34:04.630 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.630 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.630 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.631 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.631 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.631 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.631 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.631 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.632 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.632 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9540005070 con 0x7f954813abc0 2023-11-02T15:34:04.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953e7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.330+0000 7f953ffff640 4 set_mon_vals no callback set 2023-11-02T15:34:04.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f416dc40640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 0x7f41680ae1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416400e8a0 tx=0x7f416400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f415e7fc640 10 client.?.objecter ms_handle_connect 0x7f41680a84b0 2023-11-02T15:34:04.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f415e7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4014620 con 0x7f67fc142410 2023-11-02T15:34:04.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:04.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: dump: 2023-11-02T15:34:04.635 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.636 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.636 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.636 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.636 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.636 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.637 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.637 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.637 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.637 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.637 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f40147a0 con 0x7f67fc142410 2023-11-02T15:34:04.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4014b50 con 0x7f67fc142410 2023-11-02T15:34:04.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67f97fa640 10 monclient: dump: 2023-11-02T15:34:04.639 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.639 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.640 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.641 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.641 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f67faffd640 4 set_mon_vals no callback set 2023-11-02T15:34:04.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 5 monclient: authenticate success, global_id 4265 2023-11-02T15:34:04.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 10 monclient: _renew_subs 2023-11-02T15:34:04.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc13f3e0 con 0x7f67fc142410 2023-11-02T15:34:04.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 10 monclient: _renew_subs 2023-11-02T15:34:04.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.643 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc1d9b10 con 0x7f67fc142410 2023-11-02T15:34:04.643 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f6801f05640 1 librados: init done 2023-11-02T15:34:04.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540005460 con 0x7f954813abc0 2023-11-02T15:34:04.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f953e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f953e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f953e7fc640 10 monclient: dump: 2023-11-02T15:34:04.644 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.646 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.646 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.646 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.646 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 5 monclient: authenticate success, global_id 4268 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 10 monclient: _renew_subs 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95480b26e0 con 0x7f954813abc0 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 10 client.4268.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 10 monclient: _renew_subs 2023-11-02T15:34:04.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481d61c0 con 0x7f954813abc0 2023-11-02T15:34:04.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.331+0000 7f954fd0a640 1 librados: init done 2023-11-02T15:34:04.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 10 client.4268.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 15 client.4268.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 20 client.4268.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 10 client.4268.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95480aa8d0 con 0x7f954813abc0 2023-11-02T15:34:04.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 10 client.4265.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 15 client.4265.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 20 client.4265.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.332+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67fc142270 con 0x7f67fc142410 2023-11-02T15:34:04.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0011830 con 0x7f85dc0ac560 2023-11-02T15:34:04.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:04.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: dump: 2023-11-02T15:34:04.652 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.653 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.653 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.653 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.653 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.653 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.654 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.654 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.654 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.654 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 5 monclient: authenticate success, global_id 4262 2023-11-02T15:34:04.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 10 monclient: _renew_subs 2023-11-02T15:34:04.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc132950 con 0x7f85dc0ac560 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 10 client.4262.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 10 monclient: _renew_subs 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc1311c0 con 0x7f85dc0ac560 2023-11-02T15:34:04.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85e21dd640 1 librados: init done 2023-11-02T15:34:04.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0011990 con 0x7f85dc0ac560 2023-11-02T15:34:04.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85c3fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.333+0000 7f85d9ffb640 4 set_mon_vals no callback set 2023-11-02T15:34:04.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0011830 con 0x7f85dc0ac560 2023-11-02T15:34:04.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:04.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85c3fff640 10 monclient: dump: 2023-11-02T15:34:04.658 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.658 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.658 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.659 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.659 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.659 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.659 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.659 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85e21dd640 10 client.4262.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.334+0000 7f85e21dd640 15 client.4262.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f85e21dd640 20 client.4262.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f85e21dd640 10 client.4262.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f2f9ab88640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 0x7f2f941d6c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00efc0 tx=0x7f2f8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f2f8affd640 10 client.?.objecter ms_handle_connect 0x7f2f9413e520 2023-11-02T15:34:04.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f2f8affd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:04.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f85e21dd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.335+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85bc0028a0 con 0x7f85dc0ac560 2023-11-02T15:34:04.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c009ba0 con 0x7f2f9413e520 2023-11-02T15:34:04.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:04.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 10 monclient: dump: 2023-11-02T15:34:04.665 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.665 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.665 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.665 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.666 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.666 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.666 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.666 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.666 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.338+0000 7f2f9c5eb640 5 monclient: authenticate success, global_id 4290 2023-11-02T15:34:04.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 10 monclient: _renew_subs 2023-11-02T15:34:04.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f941400b0 con 0x7f2f9413e520 2023-11-02T15:34:04.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 10 client.4290.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 10 monclient: _renew_subs 2023-11-02T15:34:04.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f941d7370 con 0x7f2f9413e520 2023-11-02T15:34:04.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00f040 con 0x7f2f9413e520 2023-11-02T15:34:04.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f8affd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f9c5eb640 1 librados: init done 2023-11-02T15:34:04.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.339+0000 7f2f98b84640 4 set_mon_vals no callback set 2023-11-02T15:34:04.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c014680 con 0x7f2f9413e520 2023-11-02T15:34:04.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f8affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:04.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f8affd640 10 monclient: dump: 2023-11-02T15:34:04.670 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.671 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.672 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.672 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.672 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 10 client.4290.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 15 client.4290.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 20 client.4290.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 10 client.4290.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:04.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.340+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f940a8be0 con 0x7f2f9413e520 2023-11-02T15:34:04.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.355+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff54c003cf0 con 0x7ff5580abe50 2023-11-02T15:34:04.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.356+0000 7ff554ff9640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff54406be10 0x7ff54406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.356+0000 7ff557fff640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff54406be10 0x7ff54406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.356+0000 7ff557fff640 10 monclient: get_auth_request con 0x7ff54406be10 auth_method 0 2023-11-02T15:34:04.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.356+0000 7ff554ff9640 10 client.4303.objecter ms_handle_connect 0x7ff54406be10 2023-11-02T15:34:04.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.356+0000 7ff557fff640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff54406be10 0x7ff54406e2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5480079a0 tx=0x7ff548008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff54c042070 con 0x7ff5580abe50 2023-11-02T15:34:04.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 10 client.4303.objecter ms_dispatch 0x7ff558000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 3 client.4303.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f8c0147e0 con 0x7f2f9413e520 2023-11-02T15:34:04.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 3 client.4303.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 10 client.4303.objecter checking command 1 2023-11-02T15:34:04.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7ff5440721f0 0x7ff544074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter _get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 15 client.4303.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter _get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 15 client.4303.objecter _session_command_op_assign 4 1 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 4 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 10 client.4303.objecter _send_command 1 2023-11-02T15:34:04.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff544074cc0 con 0x7ff5440721f0 2023-11-02T15:34:04.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 20 client.4303.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff54c076a20 con 0x7ff5580abe50 2023-11-02T15:34:04.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7ff554ff9640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.360+0000 7f2f8affd640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2f7806bdc0 0x7f2f7806e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f2f9a387640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2f7806bdc0 0x7f2f7806e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f2f9a387640 10 monclient: get_auth_request con 0x7f2f7806bdc0 auth_method 0 2023-11-02T15:34:04.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640094f0 con 0x7f41680a84b0 2023-11-02T15:34:04.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:04.686 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: dump: 2023-11-02T15:34:04.686 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.686 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.687 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.687 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.688 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.688 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.689 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.689 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.689 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.690 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:04.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:04.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:04.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400c940 con 0x7f41680a84b0 2023-11-02T15:34:04.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:04.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400ccf0 con 0x7f41680a84b0 2023-11-02T15:34:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415e7fc640 10 monclient: dump: 2023-11-02T15:34:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:04.694 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.694 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:04.694 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:04.694 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:04.694 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:04.695 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:04.695 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:04.696 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:04.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 5 monclient: authenticate success, global_id 4309 2023-11-02T15:34:04.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 10 monclient: _renew_subs 2023-11-02T15:34:04.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7ff5577fe640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7ff5440721f0 0x7ff544074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41680ab9e0 con 0x7f41680a84b0 2023-11-02T15:34:04.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f415ffff640 4 set_mon_vals no callback set 2023-11-02T15:34:04.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 10 client.4309.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 10 monclient: _renew_subs 2023-11-02T15:34:04.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41680acb80 con 0x7f41680a84b0 2023-11-02T15:34:04.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7f416ec42640 1 librados: init done 2023-11-02T15:34:04.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.361+0000 7ff5577fe640 10 monclient: get_auth_request con 0x7ff5440721f0 auth_method 0 2023-11-02T15:34:04.701 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f9a387640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2f7806bdc0 0x7f2f7806e280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000f60 tx=0x7f2f900079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.701 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f8affd640 10 client.4290.objecter ms_handle_connect 0x7f2f7806bdc0 2023-11-02T15:34:04.702 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 10 client.4309.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:04.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 15 client.4309.objecter _session_command_op_assign -1 1 2023-11-02T15:34:04.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 20 client.4309.objecter _get_session osd=-1 returning homeless 2023-11-02T15:34:04.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 10 client.4309.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:04.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41680b1310 con 0x7f41680a84b0 2023-11-02T15:34:04.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2f8c059450 con 0x7f2f9413e520 2023-11-02T15:34:04.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f8affd640 10 client.4290.objecter ms_dispatch 0x7f2f94000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f8affd640 3 client.4290.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.362+0000 7f2f8affd640 3 client.4290.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 10 client.4290.objecter checking command 1 2023-11-02T15:34:04.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7f2f780721a0 0x7f2f780745c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter _get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.707 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 15 client.4290.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.707 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter _get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.707 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 15 client.4290.objecter _session_command_op_assign 6 1 2023-11-02T15:34:04.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 4 2023-11-02T15:34:04.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 10 client.4290.objecter _send_command 1 2023-11-02T15:34:04.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f78074c70 con 0x7f2f780721a0 2023-11-02T15:34:04.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f8affd640 20 client.4290.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.363+0000 7f2f99b86640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7f2f780721a0 0x7f2f780745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.364+0000 7f2f99b86640 10 monclient: get_auth_request con 0x7f2f780721a0 auth_method 0 2023-11-02T15:34:04.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.364+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f95400055c0 con 0x7f954813abc0 2023-11-02T15:34:04.711 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7ff554ff9640 10 client.4303.objecter ms_handle_connect 0x7ff5440721f0 2023-11-02T15:34:04.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7ff5577fe640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7ff5440721f0 0x7ff544074610 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7f953e7fc640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f952c06bdf0 0x7f952c06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7f954daa6640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f952c06bdf0 0x7f952c06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.715 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7f954daa6640 10 monclient: get_auth_request con 0x7f952c06bdf0 auth_method 0 2023-11-02T15:34:04.715 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7f954daa6640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f952c06bdf0 0x7f952c06e2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f9544030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.365+0000 7f953e7fc640 10 client.4268.objecter ms_handle_connect 0x7f952c06bdf0 2023-11-02T15:34:04.720 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.366+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f8c055090 con 0x7f2f9413e520 2023-11-02T15:34:04.720 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.366+0000 7f2f8affd640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.720 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.367+0000 7f2f99b86640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7f2f780721a0 0x7f2f780745c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.720 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.367+0000 7f2f8affd640 10 client.4290.objecter ms_handle_connect 0x7f2f780721a0 2023-11-02T15:34:04.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== osd.2 v2:172.21.15.174:6810/86153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9224074c70 con 0x7f92240721a0 2023-11-02T15:34:04.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f9232ffd640 10 client.4278.objecter ms_dispatch 0x7f9244000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f9232ffd640 10 client.4278.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f9232ffd640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 4 2023-11-02T15:34:04.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f9232ffd640 15 client.4278.objecter _session_command_op_remove 2 1 2023-11-02T15:34:04.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== osd.0 v2:172.21.15.174:6802/86149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5744074ca0 con 0x7f57440721d0 2023-11-02T15:34:04.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f5754ff9640 10 client.4294.objecter ms_dispatch 0x7f5758000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f5754ff9640 10 client.4294.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f5754ff9640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 4 2023-11-02T15:34:04.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.368+0000 7f5754ff9640 15 client.4294.objecter _session_command_op_remove 0 1 2023-11-02T15:34:04.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.378+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f67f4014cb0 con 0x7f67fc142410 2023-11-02T15:34:04.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.378+0000 7f67f97fa640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f67e006be10 0x7f67e006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.378+0000 7f6800f03640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f67e006be10 0x7f67e006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.378+0000 7f6800f03640 10 monclient: get_auth_request con 0x7f67e006be10 auth_method 0 2023-11-02T15:34:04.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.379+0000 7f67f97fa640 10 client.4265.objecter ms_handle_connect 0x7f67e006be10 2023-11-02T15:34:04.727 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.379+0000 7f6800f03640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f67e006be10 0x7f67e006e2d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.728 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 10 client.4278.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.728 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 15 client.4278.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.728 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 20 client.4278.objecter _get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.729 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.729 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 20 client.4278.objecter _get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 15 client.4278.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 20 client.4278.objecter get_session s=0x7f9224071ad0 osd=2 3 2023-11-02T15:34:04.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 15 client.4278.objecter _session_command_op_assign 2 2 2023-11-02T15:34:04.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 4 2023-11-02T15:34:04.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 10 client.4278.objecter _send_command 2 2023-11-02T15:34:04.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.382+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 --> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f923c0c7a80 con 0x7f92240721a0 2023-11-02T15:34:04.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 10 client.4294.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 15 client.4294.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 20 client.4294.objecter _get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 20 client.4294.objecter _get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 15 client.4294.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 20 client.4294.objecter get_session s=0x7f5744071b00 osd=0 3 2023-11-02T15:34:04.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 15 client.4294.objecter _session_command_op_assign 0 2 2023-11-02T15:34:04.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 4 2023-11-02T15:34:04.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 10 client.4294.objecter _send_command 2 2023-11-02T15:34:04.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.383+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57580afff0 con 0x7f57440721d0 2023-11-02T15:34:04.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f67f40599b0 con 0x7f67fc142410 2023-11-02T15:34:04.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 10 client.4265.objecter ms_dispatch 0x7f67fc000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 3 client.4265.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 3 client.4265.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 10 client.4265.objecter checking command 1 2023-11-02T15:34:04.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter _get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 15 client.4265.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter _get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 15 client.4265.objecter _session_command_op_assign 5 1 2023-11-02T15:34:04.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 4 2023-11-02T15:34:04.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 10 client.4265.objecter _send_command 1 2023-11-02T15:34:04.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67e0074cc0 con 0x7f67e00721f0 2023-11-02T15:34:04.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.385+0000 7f67f97fa640 20 client.4265.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.386+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.386+0000 7f67fbfff640 10 monclient: get_auth_request con 0x7f67e00721f0 auth_method 0 2023-11-02T15:34:04.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f9232ffd640 1 -- 172.21.15.174:0/2686787306 <== osd.2 v2:172.21.15.174:6810/86153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9238011040 con 0x7f92240721a0 2023-11-02T15:34:04.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f9232ffd640 10 client.4278.objecter ms_dispatch 0x7f9244000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f9232ffd640 10 client.4278.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f9232ffd640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 4 2023-11-02T15:34:04.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f9232ffd640 15 client.4278.objecter _session_command_op_remove 2 2 2023-11-02T15:34:04.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f5754ff9640 1 -- 172.21.15.174:0/2101142167 <== osd.0 v2:172.21.15.174:6802/86149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57480011c0 con 0x7f57440721d0 2023-11-02T15:34:04.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f5754ff9640 10 client.4294.objecter ms_dispatch 0x7f5758000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f5754ff9640 10 client.4294.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f5754ff9640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 4 2023-11-02T15:34:04.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f5754ff9640 15 client.4294.objecter _session_command_op_remove 0 2 2023-11-02T15:34:04.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 10 librados: watch_flush enter 2023-11-02T15:34:04.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 10 librados: watch_flush exit 2023-11-02T15:34:04.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 10 client.4278.objecter close_session for osd.2 2023-11-02T15:34:04.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f92240721a0 msgr2=0x7f92240745c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f92240721a0 0x7f92240745c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 20 client.4278.objecter put_session s=0x7f9224071ad0 osd=2 1 2023-11-02T15:34:04.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 20 client.4278.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 10 client.4278.objecter successfully canceled tick 2023-11-02T15:34:04.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f922406bdc0 msgr2=0x7f922406e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f922406bdc0 0x7f922406e280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f924404f940 tx=0x7f9244073200 comp rx=0 tx=0).stop 2023-11-02T15:34:04.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 10 monclient: shutdown 2023-11-02T15:34:04.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 msgr2=0x7f923c0c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f924359e640 1 --2- 172.21.15.174:0/2686787306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923c0c3c80 0x7f923c0c0cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f92340046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 10 librados: watch_flush enter 2023-11-02T15:34:04.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 10 librados: watch_flush exit 2023-11-02T15:34:04.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 10 client.4294.objecter close_session for osd.0 2023-11-02T15:34:04.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7f57440721d0 msgr2=0x7f57440745f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7f57440721d0 0x7f57440745f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 20 client.4294.objecter put_session s=0x7f5744071b00 osd=0 1 2023-11-02T15:34:04.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 20 client.4294.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 10 client.4294.objecter successfully canceled tick 2023-11-02T15:34:04.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.750 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f67f97fa640 10 client.4265.objecter ms_handle_connect 0x7f67e00721f0 2023-11-02T15:34:04.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f574406bdf0 msgr2=0x7f574406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f574406bdf0 0x7f574406e2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5750038000 tx=0x7f5750030000 comp rx=0 tx=0).stop 2023-11-02T15:34:04.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 10 monclient: shutdown 2023-11-02T15:34:04.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.389+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 msgr2=0x7f575813c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 --2- 172.21.15.174:0/2101142167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57580a9eb0 0x7f575813c8e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e870 tx=0x7f574c00ed30 comp rx=0 tx=0).stop 2023-11-02T15:34:04.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575cecb640 1 -- 172.21.15.174:0/2101142167 reap_dead start 2023-11-02T15:34:04.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 shutdown_connections 2023-11-02T15:34:04.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 >> 172.21.15.174:0/2101142167 conn(0x7f57580af4d0 msgr2=0x7f575813dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 shutdown_connections 2023-11-02T15:34:04.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 -- 172.21.15.174:0/2101142167 wait complete. 2023-11-02T15:34:04.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.390+0000 7f575d6cc640 1 librados: shutdown 2023-11-02T15:34:04.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f9242d9d640 1 -- 172.21.15.174:0/2686787306 reap_dead start 2023-11-02T15:34:04.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 shutdown_connections 2023-11-02T15:34:04.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 >> 172.21.15.174:0/2686787306 conn(0x7f923c0061c0 msgr2=0x7f923c02a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 shutdown_connections 2023-11-02T15:34:04.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f924359e640 1 -- 172.21.15.174:0/2686787306 wait complete. 2023-11-02T15:34:04.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f924359e640 1 librados: shutdown 2023-11-02T15:34:04.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.391+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4164013470 con 0x7f41680a84b0 2023-11-02T15:34:04.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.392+0000 7f415e7fc640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414c06be10 0x7f414c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.392+0000 7f416d43f640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414c06be10 0x7f414c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.392+0000 7f416d43f640 10 monclient: get_auth_request con 0x7f414c06be10 auth_method 0 2023-11-02T15:34:04.757 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.392+0000 7f416d43f640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414c06be10 0x7f414c06e2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4158035000 tx=0x7f4158030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.757 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.392+0000 7f415e7fc640 10 client.4309.objecter ms_handle_connect 0x7f414c06be10 2023-11-02T15:34:04.759 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9540057aa0 con 0x7f954813abc0 2023-11-02T15:34:04.759 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 10 client.4268.objecter ms_dispatch 0x7f9548000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.759 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 3 client.4268.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 3 client.4268.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 10 client.4268.objecter checking command 1 2023-11-02T15:34:04.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f952c0721d0 0x7f952c0745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 20 client.4268.objecter _get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 15 client.4268.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 20 client.4268.objecter _get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 20 client.4268.objecter get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 15 client.4268.objecter _session_command_op_assign 3 1 2023-11-02T15:34:04.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 4 2023-11-02T15:34:04.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.395+0000 7f953e7fc640 10 client.4268.objecter _send_command 1 2023-11-02T15:34:04.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.396+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f952c074ca0 con 0x7f952c0721d0 2023-11-02T15:34:04.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.396+0000 7f953e7fc640 20 client.4268.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.396+0000 7f954e2a7640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f952c0721d0 0x7f952c0745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.396+0000 7f954e2a7640 10 monclient: get_auth_request con 0x7f952c0721d0 auth_method 0 2023-11-02T15:34:04.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41640586d0 con 0x7f41680a84b0 2023-11-02T15:34:04.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 10 client.4309.objecter ms_dispatch 0x7f4168000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 3 client.4309.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 3 client.4309.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 10 client.4309.objecter checking command 1 2023-11-02T15:34:04.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f414c0721f0 0x7f414c074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter _get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 15 client.4309.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter _get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 15 client.4309.objecter _session_command_op_assign 1 1 2023-11-02T15:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 4 2023-11-02T15:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 10 client.4309.objecter _send_command 1 2023-11-02T15:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f414c074cc0 con 0x7f414c0721f0 2023-11-02T15:34:04.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.398+0000 7f415e7fc640 20 client.4309.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.399+0000 7f416e441640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f414c0721f0 0x7f414c074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.399+0000 7f416e441640 10 monclient: get_auth_request con 0x7f414c0721f0 auth_method 0 2023-11-02T15:34:04.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.401+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4164054310 con 0x7f41680a84b0 2023-11-02T15:34:04.767 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.401+0000 7f415e7fc640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.402+0000 7f954e2a7640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f952c0721d0 0x7f952c0745f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.402+0000 7f953e7fc640 10 client.4268.objecter ms_handle_connect 0x7f952c0721d0 2023-11-02T15:34:04.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.403+0000 7f416e441640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f414c0721f0 0x7f414c074610 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.403+0000 7f415e7fc640 10 client.4309.objecter ms_handle_connect 0x7f414c0721f0 2023-11-02T15:34:04.770 INFO:teuthology.orchestra.run.smithi174.stdout:68719476742 2023-11-02T15:34:04.770 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-02T15:34:04.771 INFO:teuthology.orchestra.run.smithi174.stdout:68719476741 2023-11-02T15:34:04.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-02T15:34:04.773 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.413+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f85d0011b00 con 0x7f85dc0ac560 2023-11-02T15:34:04.773 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.413+0000 7f85c3fff640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f85c806be80 0x7f85c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.773 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.413+0000 7f85db7fe640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f85c806be80 0x7f85c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.773 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f85db7fe640 10 monclient: get_auth_request con 0x7f85c806be80 auth_method 0 2023-11-02T15:34:04.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== osd.6 v2:172.21.15.175:6824/86417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2f78074c70 con 0x7f2f780721a0 2023-11-02T15:34:04.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f2f8affd640 10 client.4290.objecter ms_dispatch 0x7f2f94000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f2f8affd640 10 client.4290.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f2f8affd640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 4 2023-11-02T15:34:04.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f2f8affd640 15 client.4290.objecter _session_command_op_remove 6 1 2023-11-02T15:34:04.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f85db7fe640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f85c806be80 0x7f85c806e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000f60 tx=0x7f85cc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.414+0000 7f85c3fff640 10 client.4262.objecter ms_handle_connect 0x7f85c806be80 2023-11-02T15:34:04.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.419+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67f40555f0 con 0x7f67fc142410 2023-11-02T15:34:04.777 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.419+0000 7f67f97fa640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.777 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.419+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95400536e0 con 0x7f954813abc0 2023-11-02T15:34:04.777 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.419+0000 7f953e7fc640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f85d00584f0 con 0x7f85dc0ac560 2023-11-02T15:34:04.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 10 client.4262.objecter ms_dispatch 0x7f85dc000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:04.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 3 client.4262.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:04.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 3 client.4262.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:04.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 10 client.4262.objecter checking command 1 2023-11-02T15:34:04.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f85c8072260 0x7f85c8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter _get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 15 client.4262.objecter _session_command_op_remove -1 1 2023-11-02T15:34:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter _get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 15 client.4262.objecter _session_command_op_assign 7 1 2023-11-02T15:34:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 4 2023-11-02T15:34:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 10 client.4262.objecter _send_command 1 2023-11-02T15:34:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85c8074d30 con 0x7f85c8072260 2023-11-02T15:34:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.420+0000 7f85c3fff640 20 client.4262.objecter dump_active .. 0 homeless 2023-11-02T15:34:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.421+0000 7f85dbfff640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f85c8072260 0x7f85c8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.421+0000 7f85dbfff640 10 monclient: get_auth_request con 0x7f85c8072260 auth_method 0 2023-11-02T15:34:04.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.421+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d0011830 con 0x7f85dc0ac560 2023-11-02T15:34:04.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.421+0000 7f85c3fff640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:34:04.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.425+0000 7f85dbfff640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f85c8072260 0x7f85c8074680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.425+0000 7f85c3fff640 10 client.4262.objecter ms_handle_connect 0x7f85c8072260 2023-11-02T15:34:04.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 10 client.4290.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 15 client.4290.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 20 client.4290.objecter _get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 20 client.4290.objecter _get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 15 client.4290.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 20 client.4290.objecter get_session s=0x7f2f78071ad0 osd=6 3 2023-11-02T15:34:04.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 15 client.4290.objecter _session_command_op_assign 6 2 2023-11-02T15:34:04.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 4 2023-11-02T15:34:04.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 10 client.4290.objecter _send_command 2 2023-11-02T15:34:04.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.429+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 --> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f940ad830 con 0x7f2f780721a0 2023-11-02T15:34:04.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.436+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== osd.4 v2:172.21.15.175:6816/86415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff544074cc0 con 0x7ff5440721f0 2023-11-02T15:34:04.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.436+0000 7ff554ff9640 10 client.4303.objecter ms_dispatch 0x7ff558000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.436+0000 7ff554ff9640 10 client.4303.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.436+0000 7ff554ff9640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 4 2023-11-02T15:34:04.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.436+0000 7ff554ff9640 15 client.4303.objecter _session_command_op_remove 4 1 2023-11-02T15:34:04.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== osd.5 v2:172.21.15.175:6808/86416 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f67e0074cc0 con 0x7f67e00721f0 2023-11-02T15:34:04.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f67f97fa640 10 client.4265.objecter ms_dispatch 0x7f67fc000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f67f97fa640 10 client.4265.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f67f97fa640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 4 2023-11-02T15:34:04.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f67f97fa640 15 client.4265.objecter _session_command_op_remove 5 1 2023-11-02T15:34:04.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f2f8affd640 1 -- 172.21.15.174:0/3822503894 <== osd.6 v2:172.21.15.175:6824/86417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f940ad830 con 0x7f2f780721a0 2023-11-02T15:34:04.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f2f8affd640 10 client.4290.objecter ms_dispatch 0x7f2f94000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f2f8affd640 10 client.4290.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f2f8affd640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 4 2023-11-02T15:34:04.790 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.450+0000 7f2f8affd640 15 client.4290.objecter _session_command_op_remove 6 2 2023-11-02T15:34:04.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 10 librados: watch_flush enter 2023-11-02T15:34:04.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 10 librados: watch_flush exit 2023-11-02T15:34:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 10 client.4290.objecter close_session for osd.6 2023-11-02T15:34:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7f2f780721a0 msgr2=0x7f2f780745c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7f2f780721a0 0x7f2f780745c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 20 client.4290.objecter put_session s=0x7f2f78071ad0 osd=6 1 2023-11-02T15:34:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 20 client.4290.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 10 client.4290.objecter successfully canceled tick 2023-11-02T15:34:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2f7806bdc0 msgr2=0x7f2f7806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2f7806bdc0 0x7f2f7806e280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000f60 tx=0x7f2f900079b0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 10 monclient: shutdown 2023-11-02T15:34:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 msgr2=0x7f2f941d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 --2- 172.21.15.174:0/3822503894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f9413e520 0x7f2f941d6c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00efc0 tx=0x7f2f8c00c490 comp rx=0 tx=0).stop 2023-11-02T15:34:04.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9ab88640 1 -- 172.21.15.174:0/3822503894 reap_dead start 2023-11-02T15:34:04.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 shutdown_connections 2023-11-02T15:34:04.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 >> 172.21.15.174:0/3822503894 conn(0x7f2f94065d50 msgr2=0x7f2f9413c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 shutdown_connections 2023-11-02T15:34:04.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 -- 172.21.15.174:0/3822503894 wait complete. 2023-11-02T15:34:04.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.451+0000 7f2f9c5eb640 1 librados: shutdown 2023-11-02T15:34:04.796 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 10 client.4303.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.796 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 15 client.4303.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.796 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 20 client.4303.objecter _get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.796 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 20 client.4303.objecter _get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 15 client.4303.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 20 client.4303.objecter get_session s=0x7ff544071b20 osd=4 3 2023-11-02T15:34:04.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 15 client.4303.objecter _session_command_op_assign 4 2 2023-11-02T15:34:04.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 4 2023-11-02T15:34:04.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 10 client.4303.objecter _send_command 2 2023-11-02T15:34:04.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 --> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5380011c0 con 0x7ff5440721f0 2023-11-02T15:34:04.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== osd.1 v2:172.21.15.174:6819/86146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f414c074cc0 con 0x7f414c0721f0 2023-11-02T15:34:04.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f415e7fc640 10 client.4309.objecter ms_dispatch 0x7f4168000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f415e7fc640 10 client.4309.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f415e7fc640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 4 2023-11-02T15:34:04.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f415e7fc640 15 client.4309.objecter _session_command_op_remove 1 1 2023-11-02T15:34:04.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== osd.3 v2:172.21.15.174:6818/86148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f952c074ca0 con 0x7f952c0721d0 2023-11-02T15:34:04.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f953e7fc640 10 client.4268.objecter ms_dispatch 0x7f9548000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f953e7fc640 10 client.4268.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f953e7fc640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 4 2023-11-02T15:34:04.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.453+0000 7f953e7fc640 15 client.4268.objecter _session_command_op_remove 3 1 2023-11-02T15:34:04.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff554ff9640 1 -- 172.21.15.174:0/3140965423 <== osd.4 v2:172.21.15.175:6816/86415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5380011c0 con 0x7ff5440721f0 2023-11-02T15:34:04.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff554ff9640 10 client.4303.objecter ms_dispatch 0x7ff558000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff554ff9640 10 client.4303.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff554ff9640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 4 2023-11-02T15:34:04.802 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff554ff9640 15 client.4303.objecter _session_command_op_remove 4 2 2023-11-02T15:34:04.803 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 10 librados: watch_flush enter 2023-11-02T15:34:04.803 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 10 librados: watch_flush exit 2023-11-02T15:34:04.803 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 10 client.4303.objecter close_session for osd.4 2023-11-02T15:34:04.804 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7ff5440721f0 msgr2=0x7ff544074610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.804 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7ff5440721f0 0x7ff544074610 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.804 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 20 client.4303.objecter put_session s=0x7ff544071b20 osd=4 1 2023-11-02T15:34:04.804 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 20 client.4303.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.804 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.459+0000 7ff55e9a4640 10 client.4303.objecter successfully canceled tick 2023-11-02T15:34:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff54406be10 msgr2=0x7ff54406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7ff54406be10 0x7ff54406e2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5480079a0 tx=0x7ff548008040 comp rx=0 tx=0).stop 2023-11-02T15:34:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 10 monclient: shutdown 2023-11-02T15:34:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 msgr2=0x7ff558129150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 --2- 172.21.15.174:0/3140965423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5580abe50 0x7ff558129150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff54c02b8a0 tx=0x7ff54c0046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55cf41640 1 -- 172.21.15.174:0/3140965423 reap_dead start 2023-11-02T15:34:04.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 shutdown_connections 2023-11-02T15:34:04.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 >> 172.21.15.174:0/3140965423 conn(0x7ff558069e50 msgr2=0x7ff5580ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 shutdown_connections 2023-11-02T15:34:04.807 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 -- 172.21.15.174:0/3140965423 wait complete. 2023-11-02T15:34:04.807 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.460+0000 7ff55e9a4640 1 librados: shutdown 2023-11-02T15:34:04.808 INFO:teuthology.orchestra.run.smithi174.stdout:60129542150 2023-11-02T15:34:04.808 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-02T15:34:04.809 INFO:teuthology.orchestra.run.smithi174.stdout:60129542149 2023-11-02T15:34:04.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-02T15:34:04.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 10 client.4268.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.811 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 15 client.4268.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.811 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 20 client.4268.objecter _get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.811 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 20 client.4268.objecter _get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 15 client.4268.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 20 client.4268.objecter get_session s=0x7f952c071b00 osd=3 3 2023-11-02T15:34:04.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 15 client.4268.objecter _session_command_op_assign 3 2 2023-11-02T15:34:04.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 4 2023-11-02T15:34:04.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 10 client.4268.objecter _send_command 2 2023-11-02T15:34:04.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.474+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 --> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f954804e040 con 0x7f952c0721d0 2023-11-02T15:34:04.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 10 client.4265.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 15 client.4265.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 20 client.4265.objecter _get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 20 client.4265.objecter _get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 15 client.4265.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 20 client.4265.objecter get_session s=0x7f67e0071b20 osd=5 3 2023-11-02T15:34:04.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 15 client.4265.objecter _session_command_op_assign 5 2 2023-11-02T15:34:04.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 4 2023-11-02T15:34:04.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 10 client.4265.objecter _send_command 2 2023-11-02T15:34:04.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67fc144d00 con 0x7f67e00721f0 2023-11-02T15:34:04.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67fbfff640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 msgr2=0x7f67e0074610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-11-02T15:34:04.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67fbfff640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 msgr2=0x7f67e0074610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-02T15:34:04.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 client.4265.objecter ms_handle_reset 0x7f67e00721f0 session 0x7f67e0071b20 osd.5 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 10 client.4265.objecter reopen_session osd.5 session, addr now [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 msgr2=0x7f67e0074610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e00721f0 0x7f67e0074610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e0074ec0 0x7f67e00772b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:04.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 10 client.4265.objecter _send_command 2 2023-11-02T15:34:04.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67e0074cc0 con 0x7f67e0074ec0 2023-11-02T15:34:04.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:04.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 10 monclient: _renew_subs 2023-11-02T15:34:04.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:04.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.480+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=23}) v3 -- 0x7f67e0077890 con 0x7f67fc142410 2023-11-02T15:34:04.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.481+0000 7f953e7fc640 1 -- 172.21.15.174:0/336913061 <== osd.3 v2:172.21.15.174:6818/86148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f954804e040 con 0x7f952c0721d0 2023-11-02T15:34:04.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.481+0000 7f953e7fc640 10 client.4268.objecter ms_dispatch 0x7f9548000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.481+0000 7f953e7fc640 10 client.4268.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.481+0000 7f953e7fc640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 4 2023-11-02T15:34:04.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.481+0000 7f953e7fc640 15 client.4268.objecter _session_command_op_remove 3 2 2023-11-02T15:34:04.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 10 librados: watch_flush enter 2023-11-02T15:34:04.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 10 librados: watch_flush exit 2023-11-02T15:34:04.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 10 client.4268.objecter close_session for osd.3 2023-11-02T15:34:04.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f952c0721d0 msgr2=0x7f952c0745f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f952c0721d0 0x7f952c0745f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 10 client.4309.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 15 client.4309.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 20 client.4268.objecter put_session s=0x7f952c071b00 osd=3 1 2023-11-02T15:34:04.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 20 client.4268.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 10 client.4268.objecter successfully canceled tick 2023-11-02T15:34:04.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 20 client.4309.objecter _get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 20 client.4309.objecter _get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 15 client.4309.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 20 client.4309.objecter get_session s=0x7f414c071b20 osd=1 3 2023-11-02T15:34:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 15 client.4309.objecter _session_command_op_assign 1 2 2023-11-02T15:34:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 4 2023-11-02T15:34:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 10 client.4309.objecter _send_command 2 2023-11-02T15:34:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 --> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41680b1310 con 0x7f414c0721f0 2023-11-02T15:34:04.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f952c06bdf0 msgr2=0x7f952c06e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f952c06bdf0 0x7f952c06e2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f9544030000 comp rx=0 tx=0).stop 2023-11-02T15:34:04.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 10 monclient: shutdown 2023-11-02T15:34:04.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.829 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 msgr2=0x7f9548139e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.830 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 --2- 172.21.15.174:0/336913061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f954813abc0 0x7f9548139e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f954000c7a0 tx=0x7f954000cc60 comp rx=0 tx=0).stop 2023-11-02T15:34:04.830 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954e2a7640 1 -- 172.21.15.174:0/336913061 reap_dead start 2023-11-02T15:34:04.830 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 shutdown_connections 2023-11-02T15:34:04.830 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 >> 172.21.15.174:0/336913061 conn(0x7f9548069e50 msgr2=0x7f95481330c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 shutdown_connections 2023-11-02T15:34:04.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 -- 172.21.15.174:0/336913061 wait complete. 2023-11-02T15:34:04.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.482+0000 7f954fd0a640 1 librados: shutdown 2023-11-02T15:34:04.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.488+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e0074ec0 0x7f67e00772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:04.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f67fbfff640 10 monclient: get_auth_request con 0x7f67e0074ec0 auth_method 0 2023-11-02T15:34:04.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f415e7fc640 1 -- 172.21.15.174:0/4139576698 <== osd.1 v2:172.21.15.174:6819/86146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4160006b10 con 0x7f414c0721f0 2023-11-02T15:34:04.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f415e7fc640 10 client.4309.objecter ms_dispatch 0x7f4168000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f415e7fc640 10 client.4309.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f415e7fc640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 4 2023-11-02T15:34:04.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f415e7fc640 15 client.4309.objecter _session_command_op_remove 1 2 2023-11-02T15:34:04.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 10 librados: watch_flush enter 2023-11-02T15:34:04.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 10 librados: watch_flush exit 2023-11-02T15:34:04.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 10 client.4309.objecter close_session for osd.1 2023-11-02T15:34:04.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f414c0721f0 msgr2=0x7f414c074610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f414c0721f0 0x7f414c074610 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 20 client.4309.objecter put_session s=0x7f414c071b20 osd=1 1 2023-11-02T15:34:04.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 20 client.4309.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 10 client.4309.objecter successfully canceled tick 2023-11-02T15:34:04.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414c06be10 msgr2=0x7f414c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414c06be10 0x7f414c06e2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4158035000 tx=0x7f4158030000 comp rx=0 tx=0).stop 2023-11-02T15:34:04.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 10 monclient: shutdown 2023-11-02T15:34:04.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 msgr2=0x7f41680ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 --2- 172.21.15.174:0/4139576698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41680a84b0 0x7f41680ae1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416400e8a0 tx=0x7f416400ed60 comp rx=0 tx=0).stop 2023-11-02T15:34:04.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416e441640 1 -- 172.21.15.174:0/4139576698 reap_dead start 2023-11-02T15:34:04.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 shutdown_connections 2023-11-02T15:34:04.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 >> 172.21.15.174:0/4139576698 conn(0x7f4168065b40 msgr2=0x7f4168073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 shutdown_connections 2023-11-02T15:34:04.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 -- 172.21.15.174:0/4139576698 wait complete. 2023-11-02T15:34:04.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.489+0000 7f416ec42640 1 librados: shutdown 2023-11-02T15:34:04.840 INFO:teuthology.orchestra.run.smithi174.stdout:60129542150 2023-11-02T15:34:04.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-02T15:34:04.842 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.494+0000 7f67f97fa640 10 client.4265.objecter ms_handle_connect 0x7f67e0074ec0 2023-11-02T15:34:04.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.494+0000 7f67fbfff640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e0074ec0 0x7f67e00772b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:04.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.497+0000 7f67f97fa640 1 -- 172.21.15.174:0/699461796 <== osd.5 v2:172.21.15.175:6808/86416 1 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67e0074cc0 con 0x7f67e0074ec0 2023-11-02T15:34:04.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.497+0000 7f67f97fa640 10 client.4265.objecter ms_dispatch 0x7f67fc000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.497+0000 7f67f97fa640 10 client.4265.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.497+0000 7f67f97fa640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 4 2023-11-02T15:34:04.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.497+0000 7f67f97fa640 15 client.4265.objecter _session_command_op_remove 5 2 2023-11-02T15:34:04.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 10 librados: watch_flush enter 2023-11-02T15:34:04.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 10 librados: watch_flush exit 2023-11-02T15:34:04.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 10 client.4265.objecter close_session for osd.5 2023-11-02T15:34:04.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e0074ec0 msgr2=0x7f67e00772b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f67e0074ec0 0x7f67e00772b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 20 client.4265.objecter put_session s=0x7f67e0071b20 osd=5 1 2023-11-02T15:34:04.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 20 client.4265.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 10 client.4265.objecter successfully canceled tick 2023-11-02T15:34:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f67e006be10 msgr2=0x7f67e006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f67e006be10 0x7f67e006e2d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f00079b0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 10 monclient: shutdown 2023-11-02T15:34:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 msgr2=0x7f67fc1428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 --2- 172.21.15.174:0/699461796 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67fc142410 0x7f67fc1428a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f67f4007ec0 tx=0x7f67f400e4b0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801704640 1 -- 172.21.15.174:0/699461796 reap_dead start 2023-11-02T15:34:04.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 shutdown_connections 2023-11-02T15:34:04.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 >> 172.21.15.174:0/699461796 conn(0x7f67fc065960 msgr2=0x7f67fc0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 shutdown_connections 2023-11-02T15:34:04.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 -- 172.21.15.174:0/699461796 wait complete. 2023-11-02T15:34:04.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.498+0000 7f6801f05640 1 librados: shutdown 2023-11-02T15:34:04.850 INFO:teuthology.orchestra.run.smithi174.stdout:68719476742 2023-11-02T15:34:04.851 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-02T15:34:04.852 INFO:teuthology.orchestra.run.smithi174.stdout:60129542149 2023-11-02T15:34:04.852 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-02T15:34:04.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.685+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== osd.7 v2:172.21.15.175:6800/86418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f85c8074d30 con 0x7f85c8072260 2023-11-02T15:34:04.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.685+0000 7f85c3fff640 10 client.4262.objecter ms_dispatch 0x7f85dc000b90 command_reply(tid 1: 0 ) v1 2023-11-02T15:34:04.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.685+0000 7f85c3fff640 10 client.4262.objecter _finish_command 1 = system:0 2023-11-02T15:34:04.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.685+0000 7f85c3fff640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 4 2023-11-02T15:34:04.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.685+0000 7f85c3fff640 15 client.4262.objecter _session_command_op_remove 7 1 2023-11-02T15:34:04.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 10 client.4262.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2023-11-02T15:34:04.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 15 client.4262.objecter _session_command_op_assign -1 2 2023-11-02T15:34:04.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 20 client.4262.objecter _get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 20 client.4262.objecter _get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 15 client.4262.objecter _session_command_op_remove -1 2 2023-11-02T15:34:04.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 20 client.4262.objecter get_session s=0x7f85c8071b90 osd=7 3 2023-11-02T15:34:04.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 15 client.4262.objecter _session_command_op_assign 7 2 2023-11-02T15:34:04.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 4 2023-11-02T15:34:04.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 10 client.4262.objecter _send_command 2 2023-11-02T15:34:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.699+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 --> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85bc0011c0 con 0x7f85c8072260 2023-11-02T15:34:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85c3fff640 1 -- 172.21.15.174:0/2632788559 <== osd.7 v2:172.21.15.175:6800/86418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85bc0011c0 con 0x7f85c8072260 2023-11-02T15:34:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85c3fff640 10 client.4262.objecter ms_dispatch 0x7f85dc000b90 command_reply(tid 2: 0 ) v1 2023-11-02T15:34:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85c3fff640 10 client.4262.objecter _finish_command 2 = system:0 2023-11-02T15:34:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85c3fff640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 4 2023-11-02T15:34:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85c3fff640 15 client.4262.objecter _session_command_op_remove 7 2 2023-11-02T15:34:04.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 10 librados: watch_flush enter 2023-11-02T15:34:04.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 10 librados: watch_flush exit 2023-11-02T15:34:04.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 10 client.4262.objecter close_session for osd.7 2023-11-02T15:34:04.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f85c8072260 msgr2=0x7f85c8074680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f85c8072260 0x7f85c8074680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 20 client.4262.objecter put_session s=0x7f85c8071b90 osd=7 1 2023-11-02T15:34:04.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 20 client.4262.objecter shutdown clearing up homeless session... 2023-11-02T15:34:04.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 10 client.4262.objecter successfully canceled tick 2023-11-02T15:34:04.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f85c806be80 msgr2=0x7f85c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f85c806be80 0x7f85c806e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000f60 tx=0x7f85cc0079b0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 10 monclient: shutdown 2023-11-02T15:34:04.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:04.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 msgr2=0x7f85dc0b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:04.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 --2- 172.21.15.174:0/2632788559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85dc0ac560 0x7f85dc0b0c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85d000b3e0 tx=0x7f85d000b8a0 comp rx=0 tx=0).stop 2023-11-02T15:34:04.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85dbfff640 1 -- 172.21.15.174:0/2632788559 reap_dead start 2023-11-02T15:34:04.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 shutdown_connections 2023-11-02T15:34:04.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.722+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 >> 172.21.15.174:0/2632788559 conn(0x7f85dc069e50 msgr2=0x7f85dc137f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:04.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.723+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 shutdown_connections 2023-11-02T15:34:04.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.723+0000 7f85e21dd640 1 -- 172.21.15.174:0/2632788559 wait complete. 2023-11-02T15:34:04.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:04.723+0000 7f85e21dd640 1 librados: shutdown 2023-11-02T15:34:04.865 INFO:teuthology.orchestra.run.smithi174.stdout:55834574853 2023-11-02T15:34:04.866 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-02T15:34:05.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.042+0000 7f97f63dd640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.042+0000 7f97f63dd640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.042+0000 7f97f63dd640 10 monclient: monmap: 2023-11-02T15:34:05.048 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.048 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.048 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.044372+0000 2023-11-02T15:34:05.048 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.044372+0000 2023-11-02T15:34:05.049 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.049 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.049 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.049 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.049 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.050 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.050 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.050 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.043+0000 7f97f63dd640 1 Processor -- start 2023-11-02T15:34:05.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.043+0000 7f97f63dd640 1 -- start start 2023-11-02T15:34:05.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.043+0000 7f97f63dd640 10 monclient: init 2023-11-02T15:34:05.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2023-11-02T15:34:05.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97e8039500 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:05.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f97e80398e0 0x7f97e8035a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-a con 0x7f97e80398e0 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:05.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f97e8035f50 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:05.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e80c59b0 con 0x7f97e8039500 2023-11-02T15:34:05.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e80cc170 con 0x7f97e8035f50 2023-11-02T15:34:05.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e80c7db0 con 0x7f97e80398e0 2023-11-02T15:34:05.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.044+0000 7f97f63dd640 10 monclient(hunting): authenticate will time out at 1372.387092s 2023-11-02T15:34:05.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.048+0000 7f97f497a640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f97e8035f50 0x7f97e80c81d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:38648/0 (socket says 172.21.15.174:38648) 2023-11-02T15:34:05.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.048+0000 7f97f497a640 1 -- 172.21.15.174:0/2884696058 learned_addr learned my addr 172.21.15.174:0/2884696058 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:05.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.047+0000 7f78a4a3a640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.047+0000 7f78a4a3a640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.047+0000 7f78a4a3a640 10 monclient: monmap: 2023-11-02T15:34:05.057 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.057 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.057 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.049773+0000 2023-11-02T15:34:05.058 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.049773+0000 2023-11-02T15:34:05.058 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.058 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.058 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.058 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.059 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.059 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.059 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.059 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f78a4a3a640 1 Processor -- start 2023-11-02T15:34:05.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f78a4a3a640 1 -- start start 2023-11-02T15:34:05.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f38da177640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f38da177640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f38da177640 10 monclient: monmap: 2023-11-02T15:34:05.068 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.068 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.068 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.052627+0000 2023-11-02T15:34:05.068 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.052627+0000 2023-11-02T15:34:05.069 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.069 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.069 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.070 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.074 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.075 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.075 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.075 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.050+0000 7f78a4a3a640 10 monclient: init 2023-11-02T15:34:05.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2023-11-02T15:34:05.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f38da177640 1 Processor -- start 2023-11-02T15:34:05.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78a00a9730 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:05.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78a00aeef0 0x7f78a00abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-c con 0x7f78a00aeef0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554166264 0 0) 0x7f97e80cc170 con 0x7f97e8035f50 2023-11-02T15:34:05.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 10 monclient(hunting): _negotiate 2023-11-02T15:34:05.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 10 monclient(hunting): my global_id is 4315 2023-11-02T15:34:05.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e0003520 con 0x7f97e8035f50 2023-11-02T15:34:05.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f78a00ac390 0x7f78a0148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-a con 0x7f78a00ac390 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:05.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a00aa460 con 0x7f78a00a9730 2023-11-02T15:34:05.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a00a9b10 con 0x7f78a00ac390 2023-11-02T15:34:05.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a00af330 con 0x7f78a00aeef0 2023-11-02T15:34:05.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.051+0000 7f78a4a3a640 10 monclient(hunting): authenticate will time out at 1372.394417s 2023-11-02T15:34:05.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f38da177640 1 -- start start 2023-11-02T15:34:05.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f38da177640 10 monclient: init 2023-11-02T15:34:05.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.052+0000 7f789dd9b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78a00aeef0 0x7f78a00abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2023-11-02T15:34:05.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f38d406a940 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:05.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38d40a9730 0x7f38d40ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): picked mon.noname-c con 0x7f38d40a9730 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): picked mon.noname-b con 0x7f38d40aeef0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40aa460 con 0x7f38d406a940 2023-11-02T15:34:05.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d413b450 con 0x7f38d40a9730 2023-11-02T15:34:05.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40af380 con 0x7f38d40aeef0 2023-11-02T15:34:05.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f38da177640 10 monclient(hunting): authenticate will time out at 1372.395807s 2023-11-02T15:34:05.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f17c283b640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f17c283b640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.053+0000 7f17c283b640 10 monclient: monmap: 2023-11-02T15:34:05.095 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.096 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.096 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.055487+0000 2023-11-02T15:34:05.096 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.055487+0000 2023-11-02T15:34:05.096 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.096 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.097 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.097 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.098 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.098 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.098 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.099 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38976/0 (socket says 172.21.15.174:38976) 2023-11-02T15:34:05.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 1 -- 172.21.15.174:0/613370096 learned_addr learned my addr 172.21.15.174:0/613370096 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 10 monclient(hunting): get_auth_request con 0x7f38d40aeef0 auth_method 0 2023-11-02T15:34:05.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.054+0000 7f38d3fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78a00aeef0 0x7f78a00abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57958/0 (socket says 172.21.15.174:57958) 2023-11-02T15:34:05.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 1 -- 172.21.15.174:0/2453537301 learned_addr learned my addr 172.21.15.174:0/2453537301 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 10 monclient(hunting): get_auth_request con 0x7f78a00aeef0 auth_method 0 2023-11-02T15:34:05.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f789dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f17c283b640 1 Processor -- start 2023-11-02T15:34:05.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f38d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f38d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f38d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f17c283b640 1 -- start start 2023-11-02T15:34:05.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.055+0000 7f17c283b640 10 monclient: init 2023-11-02T15:34:05.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835117468 0 0) 0x7f97e0003520 con 0x7f97e8035f50 2023-11-02T15:34:05.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 >> v2:172.21.15.174:3300/0 conn(0x7f97e80398e0 msgr2=0x7f97e8035a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 --2- 172.21.15.174:0/2884696058 >> v2:172.21.15.174:3300/0 conn(0x7f97e80398e0 0x7f97e8035a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 >> v1:172.21.15.174:6789/0 conn(0x7f97e8039500 legacy=0x7f97e80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 monclient: found mon.noname-c-legacy 2023-11-02T15:34:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.175:6789/0 2023-11-02T15:34:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8037780 con 0x7f97e8035f50 2023-11-02T15:34:05.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 692219148 0 0) 0x7f97f0074aa0 con 0x7f97e8035f50 2023-11-02T15:34:05.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2023-11-02T15:34:05.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: dump: 2023-11-02T15:34:05.108 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.109 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.109 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.109 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.109 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.109 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.110 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.110 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.110 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.110 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] but i'm connected to v1:172.21.15.175:6789/0, reconnecting 2023-11-02T15:34:05.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 >> v1:172.21.15.175:6789/0 conn(0x7f97e8035f50 legacy=0x7f97e80c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T15:34:05.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e0003970 0x7f97e0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): picked mon.a con 0x7f97e0003970 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): picked mon.b con 0x7f97e00047c0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e00051d0 0x7f97e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): picked mon.c con 0x7f97e00051d0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80c7db0 con 0x7f97e0003970 2023-11-02T15:34:05.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e000c100 con 0x7f97e00047c0 2023-11-02T15:34:05.114 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e000c2c0 con 0x7f97e00051d0 2023-11-02T15:34:05.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f97eeffd640 10 monclient(hunting): _finish_auth 0 2023-11-02T15:34:05.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f17c283b640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f17c283b640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2023-11-02T15:34:05.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f17c283b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.056+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-b con 0x7f17b4035990 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f17b402a960 0x7f17b40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-c con 0x7f17b402a960 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.122 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f17b40c7b00 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:05.122 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b402b850 con 0x7f17b40c7b00 2023-11-02T15:34:05.122 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.122 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b40333b0 con 0x7f17b402a960 2023-11-02T15:34:05.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b402ce00 con 0x7f17b4035990 2023-11-02T15:34:05.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f17c283b640 10 monclient(hunting): authenticate will time out at 1372.399991s 2023-11-02T15:34:05.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.057+0000 7f38d2ffd640 1 --2- 172.21.15.174:0/613370096 >> v2:172.21.15.175:3300/0 conn(0x7f38d40a9730 0x7f38d40ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 10 monclient(hunting): handle_auth_done global_id 4271 payload 274 2023-11-02T15:34:05.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 1 -- 172.21.15.174:0/613370096 >> v2:172.21.15.175:3300/0 conn(0x7f38d40a9730 msgr2=0x7f38d40ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 1 --2- 172.21.15.174:0/613370096 >> v2:172.21.15.175:3300/0 conn(0x7f38d40a9730 0x7f38d40ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 1 -- 172.21.15.174:0/613370096 >> v1:172.21.15.175:6789/0 conn(0x7f38d406a940 legacy=0x7f38d40abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 1 monclient: found mon.noname-b 2023-11-02T15:34:05.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.058+0000 7f38d3fff640 1 -- 172.21.15.174:0/613370096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40aa8f0 con 0x7f38d40aeef0 2023-11-02T15:34:05.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 10 monclient(hunting): handle_auth_done global_id 4318 payload 274 2023-11-02T15:34:05.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 1 -- 172.21.15.174:0/2453537301 >> v2:172.21.15.174:3300/0 conn(0x7f78a00ac390 msgr2=0x7f78a0148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f97f497a640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e00051d0 0x7f97e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 1 --2- 172.21.15.174:0/2453537301 >> v2:172.21.15.174:3300/0 conn(0x7f78a00ac390 0x7f78a0148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 1 -- 172.21.15.174:0/2453537301 >> v1:172.21.15.174:6789/0 conn(0x7f78a00a9730 legacy=0x7f78a013abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f789dd9b640 1 -- 172.21.15.174:0/2453537301 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00aa8f0 con 0x7f78a00aeef0 2023-11-02T15:34:05.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f65ac4db640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f65ac4db640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.060+0000 7f65ac4db640 10 monclient: monmap: 2023-11-02T15:34:05.137 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.137 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.137 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.062708+0000 2023-11-02T15:34:05.137 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.062708+0000 2023-11-02T15:34:05.138 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.138 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.138 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.138 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.138 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.139 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.139 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.139 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f65ac4db640 1 Processor -- start 2023-11-02T15:34:05.140 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f65ac4db640 1 -- start start 2023-11-02T15:34:05.140 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f65ac4db640 10 monclient: init 2023-11-02T15:34:05.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f5fb3fff640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f5fb3fff640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.061+0000 7f5fb3fff640 10 monclient: monmap: 2023-11-02T15:34:05.142 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.142 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.143 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.063767+0000 2023-11-02T15:34:05.143 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.063767+0000 2023-11-02T15:34:05.143 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.143 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.143 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.144 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.144 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.144 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.144 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.145 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.062+0000 7f97f497a640 10 monclient(hunting): get_auth_request con 0x7f97e00051d0 auth_method 0 2023-11-02T15:34:05.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.062+0000 7f97f497a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.062+0000 7f97f497a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.062+0000 7f97f497a640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:34:05.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.062+0000 7f17c1839640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 Processor -- start 2023-11-02T15:34:05.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient: _add_conns ranks=[4,2,0,1,3] 2023-11-02T15:34:05.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65a40b3da0 0x7f65a4141720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.150 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 -- start start 2023-11-02T15:34:05.150 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient: init 2023-11-02T15:34:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-c con 0x7f65a40b3da0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a4141c60 0x7f65a4144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-b con 0x7f65a4141c60 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f65a41451e0 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a40a8c20 con 0x7f65a41451e0 2023-11-02T15:34:05.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40b4180 con 0x7f65a40b3da0 2023-11-02T15:34:05.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40a84b0 con 0x7f65a4141c60 2023-11-02T15:34:05.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f65ac4db640 10 monclient(hunting): authenticate will time out at 1372.406166s 2023-11-02T15:34:05.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f789dd9b640 1 --2- 172.21.15.174:0/2453537301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00aeef0 0x7f78a00abe50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78a00ab680 tx=0x7f789002cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463a735935ed315c server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient: _add_conns ranks=[2,3,4,1,0] 2023-11-02T15:34:05.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f5fac0073f0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5fac0c5370 0x7f5fac0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f5fac0c5370 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:05.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f38d3fff640 1 --2- 172.21.15.174:0/613370096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009e50 tx=0x7f38cc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7154b4fd1b47e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fac0cb9f0 0x7f5fac036ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f5fac0cb9f0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac0094e0 con 0x7f5fac0c5370 2023-11-02T15:34:05.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac0077d0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac009200 con 0x7f5fac0073f0 2023-11-02T15:34:05.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.063+0000 7f5fb3fff640 10 monclient(hunting): authenticate will time out at 1372.406646s 2023-11-02T15:34:05.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38982/0 (socket says 172.21.15.174:38982) 2023-11-02T15:34:05.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 1 -- 172.21.15.174:0/2113404790 learned_addr learned my addr 172.21.15.174:0/2113404790 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 10 monclient(hunting): get_auth_request con 0x7f17b4035990 auth_method 0 2023-11-02T15:34:05.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.064+0000 7f17c1839640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f97f497a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.163 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f97f497a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.163 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f97f497a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-11-02T15:34:05.163 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.065+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f65aa277640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65a40b3da0 0x7f65a4141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2023-11-02T15:34:05.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 1 -- 172.21.15.174:0/2113404790 >> v2:172.21.15.175:3300/0 conn(0x7f17b402a960 msgr2=0x7f17b40c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 1 --2- 172.21.15.174:0/2113404790 >> v2:172.21.15.175:3300/0 conn(0x7f17b402a960 0x7f17b40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 1 -- 172.21.15.174:0/2113404790 >> v1:172.21.15.174:6789/0 conn(0x7f17b40c7b00 legacy=0x7f17b40c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 1 monclient: found mon.noname-b 2023-11-02T15:34:05.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f17c1839640 1 -- 172.21.15.174:0/2113404790 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40339d0 con 0x7f17b4035990 2023-11-02T15:34:05.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.067+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65a40b3da0 0x7f65a4141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:57992/0 (socket says 172.21.15.174:57992) 2023-11-02T15:34:05.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f97ef7fe640 10 monclient(hunting): get_auth_request con 0x7f97e00047c0 auth_method 0 2023-11-02T15:34:05.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f97ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f97ef7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f97ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:34:05.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 1 -- 172.21.15.174:0/3963528259 learned_addr learned my addr 172.21.15.174:0/3963528259 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 10 monclient(hunting): get_auth_request con 0x7f65a40b3da0 auth_method 0 2023-11-02T15:34:05.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.068+0000 7f65aa277640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.171 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.069+0000 7f97ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.069+0000 7f97ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.069+0000 7f97ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-11-02T15:34:05.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 10 monclient(hunting): handle_auth_done global_id 4315 payload 293 2023-11-02T15:34:05.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 -- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e00051d0 msgr2=0x7f97e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e00051d0 0x7f97e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 -- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e0003970 msgr2=0x7f97e0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e0003970 0x7f97e0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 monclient: found mon.b 2023-11-02T15:34:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 1 -- 172.21.15.174:0/2884696058 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8037780 con 0x7f97e00047c0 2023-11-02T15:34:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97f497a640 1 -- 172.21.15.174:0/2884696058 reap_dead start 2023-11-02T15:34:05.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97f63dd640 5 monclient: authenticate success, global_id 4315 2023-11-02T15:34:05.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.071+0000 7f97f63dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.072+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.072+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.072+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.075+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e0004c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009860 tx=0x7f97e00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2819095e674583f server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.076+0000 7f5fb2ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.077+0000 7f17c1839640 1 --2- 172.21.15.174:0/2113404790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009870 tx=0x7f17ac02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf2f4d55b8769f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:39012/0 (socket says 172.21.15.174:39012) 2023-11-02T15:34:05.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb37fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fac0cb9f0 0x7f5fac036ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 1 -- 172.21.15.174:0/1027473890 learned_addr learned my addr 172.21.15.174:0/1027473890 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fac0073f0 auth_method 0 2023-11-02T15:34:05.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 1 -- 172.21.15.174:0/613370096 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc02ca70 con 0x7f38d40aeef0 2023-11-02T15:34:05.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:05.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: dump: 2023-11-02T15:34:05.182 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.182 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.182 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.182 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.182 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.183 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.183 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.183 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.183 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.183 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38da177640 5 monclient: authenticate success, global_id 4271 2023-11-02T15:34:05.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38da177640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 1 -- 172.21.15.174:0/613370096 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc02cbd0 con 0x7f38d40aeef0 2023-11-02T15:34:05.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38d27fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38da177640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.078+0000 7f38da177640 4 set_mon_vals no callback set 2023-11-02T15:34:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f5fb2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38d27fc640 1 -- 172.21.15.174:0/613370096 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc02ca70 con 0x7f38d40aeef0 2023-11-02T15:34:05.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38d27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38d27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:05.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38d27fc640 10 monclient: dump: 2023-11-02T15:34:05.188 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.188 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.188 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.188 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.188 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.189 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.189 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.189 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.189 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.189 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65a9a76640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a4141c60 0x7f65a4144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 10 monclient(hunting): handle_auth_done global_id 4321 payload 274 2023-11-02T15:34:05.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 1 -- 172.21.15.174:0/3963528259 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a4141c60 msgr2=0x7f65a4144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a4141c60 0x7f65a4144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 1 -- 172.21.15.174:0/3963528259 >> v1:172.21.15.174:6789/0 conn(0x7f65a41451e0 legacy=0x7f65a413e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f65aa277640 1 -- 172.21.15.174:0/3963528259 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40b3810 con 0x7f65a40b3da0 2023-11-02T15:34:05.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 10 monclient: shutdown 2023-11-02T15:34:05.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 -- 172.21.15.174:0/613370096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 msgr2=0x7f38d4148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 --2- 172.21.15.174:0/613370096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009e50 tx=0x7f38cc02bed0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 -- 172.21.15.174:0/613370096 shutdown_connections 2023-11-02T15:34:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 --2- 172.21.15.174:0/613370096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40aeef0 0x7f38d4148110 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 --2- 172.21.15.174:0/613370096 >> v2:172.21.15.175:3300/0 conn(0x7f38d40a9730 0x7f38d40ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.079+0000 7f38da177640 1 -- 172.21.15.174:0/613370096 >> 172.21.15.174:0/613370096 conn(0x7f38d4069e50 msgr2=0x7f38d406a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f38da177640 1 -- 172.21.15.174:0/613370096 shutdown_connections 2023-11-02T15:34:05.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f38da177640 1 -- 172.21.15.174:0/613370096 wait complete. 2023-11-02T15:34:05.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request con 0x7f5fac0cb9f0 auth_method 0 2023-11-02T15:34:05.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.080+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 10 monclient: monmap: 2023-11-02T15:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.200 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.200 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.200 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.200 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.200 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.201 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.201 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 librados: starting msgr at 2023-11-02T15:34:05.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 librados: starting objecter 2023-11-02T15:34:05.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 1 -- 172.21.15.174:0/2453537301 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789002d9f0 con 0x7f78a00aeef0 2023-11-02T15:34:05.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: dump: 2023-11-02T15:34:05.203 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.204 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.204 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.204 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.205 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.205 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.205 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.205 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.205 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.206 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f78a4a3a640 5 monclient: authenticate success, global_id 4318 2023-11-02T15:34:05.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f78a4a3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 1 -- 172.21.15.174:0/2453537301 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789002db50 con 0x7f78a00aeef0 2023-11-02T15:34:05.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f789d59a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f78a4a3a640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f78a4a3a640 4 set_mon_vals no callback set 2023-11-02T15:34:05.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 Processor -- start 2023-11-02T15:34:05.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 -- start start 2023-11-02T15:34:05.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 librados: setting wanted keys 2023-11-02T15:34:05.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 1 librados: calling monclient init 2023-11-02T15:34:05.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.081+0000 7f38da177640 10 monclient: init 2023-11-02T15:34:05.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f789d59a640 1 -- 172.21.15.174:0/2453537301 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890035440 con 0x7f78a00aeef0 2023-11-02T15:34:05.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f789d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f789d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f789d59a640 10 monclient: dump: 2023-11-02T15:34:05.212 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.213 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.214 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.214 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.214 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:34:05.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 0x7f38d41cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 10 monclient: shutdown 2023-11-02T15:34:05.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 1 -- 172.21.15.174:0/2453537301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00aeef0 msgr2=0x7f78a00abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/2453537301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00aeef0 0x7f78a00abe50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78a00ab680 tx=0x7f789002cc40 comp rx=0 tx=0).stop 2023-11-02T15:34:05.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): picked mon.noname-b con 0x7f38d406a940 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 0x7f38d40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): picked mon.noname-c con 0x7f38d40a9730 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d40aeef0 0x7f38d40b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): picked mon.noname-a con 0x7f38d40aeef0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 1 -- 172.21.15.174:0/2453537301 shutdown_connections 2023-11-02T15:34:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/2453537301 >> v2:172.21.15.174:3300/0 conn(0x7f78a00ac390 0x7f78a0148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/2453537301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00aeef0 0x7f78a00abe50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.083+0000 7f78a4a3a640 1 -- 172.21.15.174:0/2453537301 >> 172.21.15.174:0/2453537301 conn(0x7f78a0069e50 msgr2=0x7f78a006a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40b25a0 con 0x7f38d40aeef0 2023-11-02T15:34:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.083+0000 7f78a4a3a640 1 -- 172.21.15.174:0/2453537301 shutdown_connections 2023-11-02T15:34:05.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.083+0000 7f78a4a3a640 1 -- 172.21.15.174:0/2453537301 wait complete. 2023-11-02T15:34:05.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40b1d60 con 0x7f38d406a940 2023-11-02T15:34:05.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4133d90 con 0x7f38d40a9730 2023-11-02T15:34:05.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.082+0000 7f38da177640 10 monclient(hunting): authenticate will time out at 1372.425347s 2023-11-02T15:34:05.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_done global_id 4324 payload 274 2023-11-02T15:34:05.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 -- 172.21.15.174:0/1027473890 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 msgr2=0x7f5fac0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/1027473890 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 -- 172.21.15.174:0/1027473890 >> v2:172.21.15.174:3300/0 conn(0x7f5fac0c5370 msgr2=0x7f5fac0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/1027473890 >> v2:172.21.15.174:3300/0 conn(0x7f5fac0c5370 0x7f5fac0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f5fb37fe640 1 -- 172.21.15.174:0/1027473890 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0c57b0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f78a4a3a640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f78a4a3a640 10 monclient: monmap: 2023-11-02T15:34:05.228 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.228 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.228 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.228 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.228 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.229 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.229 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.229 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.229 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.229 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f78a4a3a640 1 librados: starting msgr at 2023-11-02T15:34:05.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f78a4a3a640 1 librados: starting objecter 2023-11-02T15:34:05.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.084+0000 7f78a4a3a640 1 Processor -- start 2023-11-02T15:34:05.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 -- start start 2023-11-02T15:34:05.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 librados: setting wanted keys 2023-11-02T15:34:05.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 librados: calling monclient init 2023-11-02T15:34:05.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient: init 2023-11-02T15:34:05.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 1 -- 172.21.15.174:0/2113404790 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac031640 con 0x7f17b4035990 2023-11-02T15:34:05.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:05.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: dump: 2023-11-02T15:34:05.233 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.235 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.235 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.235 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.235 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 1 -- 172.21.15.174:0/2113404790 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac0317a0 con 0x7f17b4035990 2023-11-02T15:34:05.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 1 -- 172.21.15.174:0/2113404790 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac031b30 con 0x7f17b4035990 2023-11-02T15:34:05.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c283b640 5 monclient: authenticate success, global_id 4274 2023-11-02T15:34:05.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:05.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c0837640 10 monclient: dump: 2023-11-02T15:34:05.238 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.238 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.238 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.238 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.239 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.239 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.239 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.239 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.239 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.240 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c283b640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f17c283b640 4 set_mon_vals no callback set 2023-11-02T15:34:05.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:34:05.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 0x7f78a013c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-b con 0x7f78a00a9730 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 0x7f78a01369a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-c con 0x7f78a00ac390 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78a00aeef0 0x7f78a013d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): picked mon.noname-a con 0x7f78a00aeef0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00b2330 con 0x7f78a00aeef0 2023-11-02T15:34:05.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a013d260 con 0x7f78a00a9730 2023-11-02T15:34:05.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a013b660 con 0x7f78a00ac390 2023-11-02T15:34:05.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.085+0000 7f78a4a3a640 10 monclient(hunting): authenticate will time out at 1372.428606s 2023-11-02T15:34:05.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc032600 con 0x7f97e00047c0 2023-11-02T15:34:05.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: dump: 2023-11-02T15:34:05.247 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.247 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.247 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.247 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.248 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.248 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.248 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.248 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.248 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.249 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc02e050 con 0x7f97e00047c0 2023-11-02T15:34:05.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97f63dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 1 -- 172.21.15.174:0/2884696058 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc032a00 con 0x7f97e00047c0 2023-11-02T15:34:05.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97eeffd640 10 monclient: dump: 2023-11-02T15:34:05.250 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.250 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.251 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.251 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.251 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.251 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.251 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.252 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.252 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.252 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97f63dd640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f97f63dd640 4 set_mon_vals no callback set 2023-11-02T15:34:05.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 10 monclient: shutdown 2023-11-02T15:34:05.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 -- 172.21.15.174:0/2113404790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 msgr2=0x7f17b402bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 --2- 172.21.15.174:0/2113404790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009870 tx=0x7f17ac02b8f0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 -- 172.21.15.174:0/2113404790 shutdown_connections 2023-11-02T15:34:05.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 --2- 172.21.15.174:0/2113404790 >> v2:172.21.15.175:3300/0 conn(0x7f17b402a960 0x7f17b40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 --2- 172.21.15.174:0/2113404790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b4035990 0x7f17b402bd30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 -- 172.21.15.174:0/2113404790 >> 172.21.15.174:0/2113404790 conn(0x7f17b40064f0 msgr2=0x7f17b4006900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 -- 172.21.15.174:0/2113404790 shutdown_connections 2023-11-02T15:34:05.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.086+0000 7f17c283b640 1 -- 172.21.15.174:0/2113404790 wait complete. 2023-11-02T15:34:05.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 10 monclient: shutdown 2023-11-02T15:34:05.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 -- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 msgr2=0x7f97e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e0004c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009860 tx=0x7f97e00057f0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 -- 172.21.15.174:0/2884696058 shutdown_connections 2023-11-02T15:34:05.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 --2- 172.21.15.174:0/2884696058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e0004c40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 -- 172.21.15.174:0/2884696058 >> 172.21.15.174:0/2884696058 conn(0x7f97e8007090 msgr2=0x7f97e8037de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 -- 172.21.15.174:0/2884696058 shutdown_connections 2023-11-02T15:34:05.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f97f63dd640 1 -- 172.21.15.174:0/2884696058 wait complete. 2023-11-02T15:34:05.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f65aa277640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40b3da0 0x7f65a4141720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a23a86f0dac579 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f17c283b640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.087+0000 7f17c283b640 10 monclient: monmap: 2023-11-02T15:34:05.259 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.259 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.259 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.261 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.261 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 librados: starting msgr at 2023-11-02T15:34:05.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 librados: starting objecter 2023-11-02T15:34:05.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 Processor -- start 2023-11-02T15:34:05.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f97f63dd640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f97f63dd640 10 monclient: monmap: 2023-11-02T15:34:05.262 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.262 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.263 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.263 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.263 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.263 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.263 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.264 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.264 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.264 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 -- start start 2023-11-02T15:34:05.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 0x7f38d41cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 librados: setting wanted keys 2023-11-02T15:34:05.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 1 librados: calling monclient init 2023-11-02T15:34:05.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f17c283b640 10 monclient: init 2023-11-02T15:34:05.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f97f63dd640 1 librados: starting msgr at 2023-11-02T15:34:05.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f97f63dd640 1 librados: starting objecter 2023-11-02T15:34:05.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.088+0000 7f97f63dd640 1 Processor -- start 2023-11-02T15:34:05.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:05.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 1 -- start start 2023-11-02T15:34:05.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 1 librados: setting wanted keys 2023-11-02T15:34:05.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 1 librados: calling monclient init 2023-11-02T15:34:05.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 10 monclient: init 2023-11-02T15:34:05.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-c con 0x7f17b402a960 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b4035990 0x7f17b40c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-b con 0x7f17b4035990 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b40c7b00 0x7f17b40be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): picked mon.noname-a con 0x7f17b40c7b00 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b415ad60 con 0x7f17b40c7b00 2023-11-02T15:34:05.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40326a0 con 0x7f17b4035990 2023-11-02T15:34:05.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b415aa30 con 0x7f17b402a960 2023-11-02T15:34:05.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f17c283b640 10 monclient(hunting): authenticate will time out at 1372.431990s 2023-11-02T15:34:05.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/1027473890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac036ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4068260 tx=0x7f5fb40a5430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb13acf1a54f9b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:05.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.089+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-b con 0x7f97e00047c0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e802d590 0x7f97e80c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-a con 0x7f97e802d590 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e80c0bc0 0x7f97e815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): picked mon.noname-c con 0x7f97e80c0bc0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8034390 con 0x7f97e802d590 2023-11-02T15:34:05.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80c1040 con 0x7f97e00047c0 2023-11-02T15:34:05.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8033e70 con 0x7f97e80c0bc0 2023-11-02T15:34:05.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f97f63dd640 10 monclient(hunting): authenticate will time out at 1372.432715s 2023-11-02T15:34:05.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 0x7f38d41cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:58008/0 (socket says 172.21.15.174:58008) 2023-11-02T15:34:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 1 -- 172.21.15.174:0/3224807096 learned_addr learned my addr 172.21.15.174:0/3224807096 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 10 monclient(hunting): get_auth_request con 0x7f38d406a940 auth_method 0 2023-11-02T15:34:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.090+0000 7f38d37fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.092+0000 7f38d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.092+0000 7f38d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.092+0000 7f38d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.092+0000 7f789e59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 0x7f78a013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.093+0000 7f38d2ffd640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 0x7f38d40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 0x7f78a01369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f38d2ffd640 10 monclient(hunting): get_auth_request con 0x7f38d40a9730 auth_method 0 2023-11-02T15:34:05.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f38d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f38d2ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f38d2ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f17c1839640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.282 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 0x7f78a01369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:39038/0 (socket says 172.21.15.174:39038) 2023-11-02T15:34:05.283 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 1 -- 172.21.15.174:0/3329458577 learned_addr learned my addr 172.21.15.174:0/3329458577 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.283 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 10 monclient(hunting): get_auth_request con 0x7f78a00ac390 auth_method 0 2023-11-02T15:34:05.283 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.283 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.283 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f789dd9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.094+0000 7f38d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f38d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f38d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 1 -- 172.21.15.174:0/3963528259 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0031640 con 0x7f65a40b3da0 2023-11-02T15:34:05.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: dump: 2023-11-02T15:34:05.286 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.286 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.286 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.286 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.286 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.287 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.287 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.287 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.287 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.287 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 1 -- 172.21.15.174:0/3963528259 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00317a0 con 0x7f65a40b3da0 2023-11-02T15:34:05.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 1 -- 172.21.15.174:0/3963528259 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0031b30 con 0x7f65a40b3da0 2023-11-02T15:34:05.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65a9275640 10 monclient: dump: 2023-11-02T15:34:05.289 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.290 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.291 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.291 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.291 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65ac4db640 5 monclient: authenticate success, global_id 4321 2023-11-02T15:34:05.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65ac4db640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f65ac4db640 4 set_mon_vals no callback set 2023-11-02T15:34:05.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f21521c9640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f21521c9640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f21521c9640 10 monclient: monmap: 2023-11-02T15:34:05.293 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.293 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.294 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.097291+0000 2023-11-02T15:34:05.294 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.097291+0000 2023-11-02T15:34:05.294 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.294 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.294 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.295 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.295 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.295 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.295 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.295 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f789e59c640 10 monclient(hunting): get_auth_request con 0x7f78a00a9730 auth_method 0 2023-11-02T15:34:05.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f789e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f789e59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.095+0000 7f789e59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 10 monclient: shutdown 2023-11-02T15:34:05.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 -- 172.21.15.174:0/3963528259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40b3da0 msgr2=0x7f65a4141720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40b3da0 0x7f65a4141720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009870 tx=0x7f65a002b8f0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 -- 172.21.15.174:0/3963528259 shutdown_connections 2023-11-02T15:34:05.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a4141c60 0x7f65a4144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f21521c9640 1 Processor -- start 2023-11-02T15:34:05.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 --2- 172.21.15.174:0/3963528259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40b3da0 0x7f65a4141720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 -- 172.21.15.174:0/3963528259 >> 172.21.15.174:0/3963528259 conn(0x7f65a4065960 msgr2=0x7f65a4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f21521c9640 1 -- start start 2023-11-02T15:34:05.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f21521c9640 10 monclient: init 2023-11-02T15:34:05.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 -- 172.21.15.174:0/3963528259 shutdown_connections 2023-11-02T15:34:05.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.096+0000 7f65ac4db640 1 -- 172.21.15.174:0/3963528259 wait complete. 2023-11-02T15:34:05.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f789e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f789e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f789e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f65ac4db640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f65ac4db640 10 monclient: monmap: 2023-11-02T15:34:05.302 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.302 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.303 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.303 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.303 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.303 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.303 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.304 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.304 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.304 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2023-11-02T15:34:05.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f214c0b4570 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:05.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f214c0b0ed0 0x7f214c144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-a con 0x7f214c0b0ed0 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:05.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f214c13e520 0x7f214c13e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-c con 0x7f214c13e520 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0a9780 con 0x7f214c0b4570 2023-11-02T15:34:05.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0b4950 con 0x7f214c0b0ed0 2023-11-02T15:34:05.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0aa1d0 con 0x7f214c13e520 2023-11-02T15:34:05.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f21521c9640 10 monclient(hunting): authenticate will time out at 1372.440132s 2023-11-02T15:34:05.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f65ac4db640 1 librados: starting msgr at 2023-11-02T15:34:05.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f65ac4db640 1 librados: starting objecter 2023-11-02T15:34:05.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.097+0000 7f65ac4db640 1 Processor -- start 2023-11-02T15:34:05.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 -- start start 2023-11-02T15:34:05.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 librados: setting wanted keys 2023-11-02T15:34:05.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 librados: calling monclient init 2023-11-02T15:34:05.309 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient: init 2023-11-02T15:34:05.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 10 monclient(hunting): handle_auth_done global_id 4280 payload 824 2023-11-02T15:34:05.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d3fff640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d40aeef0 0x7f38d40b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 -- 172.21.15.174:0/3224807096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 msgr2=0x7f38d41cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 0x7f38d41cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 -- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d40aeef0 msgr2=0x7f38d40b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d40aeef0 0x7f38d40b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f38d2ffd640 1 -- 172.21.15.174:0/3224807096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009b30 con 0x7f38d40a9730 2023-11-02T15:34:05.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:05.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 0x7f65a41d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-c con 0x7f65a40b3da0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a4141c60 0x7f65a40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-a con 0x7f65a4141c60 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 0x7f65a41d3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): picked mon.noname-b con 0x7f65a41451e0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40b0850 con 0x7f65a4141c60 2023-11-02T15:34:05.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40b0520 con 0x7f65a41451e0 2023-11-02T15:34:05.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40ac910 con 0x7f65a40b3da0 2023-11-02T15:34:05.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.098+0000 7f65ac4db640 10 monclient(hunting): authenticate will time out at 1372.441319s 2023-11-02T15:34:05.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 1 -- 172.21.15.174:0/1027473890 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40aa510 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: dump: 2023-11-02T15:34:05.318 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.318 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.318 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.318 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.319 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.319 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.319 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.319 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.319 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.320 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 5 monclient: authenticate success, global_id 4324 2023-11-02T15:34:05.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 1 -- 172.21.15.174:0/1027473890 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb40a8050 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:39052/0 (socket says 172.21.15.174:39052) 2023-11-02T15:34:05.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 1 -- 172.21.15.174:0/1498947942 learned_addr learned my addr 172.21.15.174:0/1498947942 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 10 monclient(hunting): get_auth_request con 0x7f17b402a960 auth_method 0 2023-11-02T15:34:05.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f17c1839640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 4 set_mon_vals no callback set 2023-11-02T15:34:05.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 1 -- 172.21.15.174:0/1027473890 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40aa930 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 10 monclient: shutdown 2023-11-02T15:34:05.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb1ffb640 10 monclient: dump: 2023-11-02T15:34:05.326 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.326 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.326 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.326 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.326 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.327 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.327 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.327 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.327 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.327 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 1 -- 172.21.15.174:0/1027473890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0cb9f0 msgr2=0x7f5fac036ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.099+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/1027473890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac036ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4068260 tx=0x7f5fb40a5430 comp rx=0 tx=0).stop 2023-11-02T15:34:05.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 -- 172.21.15.174:0/1027473890 shutdown_connections 2023-11-02T15:34:05.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/1027473890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac036ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/1027473890 >> v2:172.21.15.174:3300/0 conn(0x7f5fac0c5370 0x7f5fac0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/1027473890 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 -- 172.21.15.174:0/1027473890 >> 172.21.15.174:0/1027473890 conn(0x7f5fac006240 msgr2=0x7f5fac006650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 -- 172.21.15.174:0/1027473890 shutdown_connections 2023-11-02T15:34:05.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 1 -- 172.21.15.174:0/1027473890 wait complete. 2023-11-02T15:34:05.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.100+0000 7f5fb3fff640 10 monclient: monmap: 2023-11-02T15:34:05.330 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.330 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.330 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.331 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.331 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.331 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.331 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.331 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.332 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.332 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 librados: starting msgr at 2023-11-02T15:34:05.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 librados: starting objecter 2023-11-02T15:34:05.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 Processor -- start 2023-11-02T15:34:05.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f789dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 -- start start 2023-11-02T15:34:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 librados: setting wanted keys 2023-11-02T15:34:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 1 librados: calling monclient init 2023-11-02T15:34:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.101+0000 7f5fb3fff640 10 monclient: init 2023-11-02T15:34:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f17c1839640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 10 monclient(hunting): handle_auth_done global_id 4330 payload 824 2023-11-02T15:34:05.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 -- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 msgr2=0x7f78a01369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 0x7f78a01369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 -- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78a00aeef0 msgr2=0x7f78a013d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78a00aeef0 0x7f78a013d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 monclient: found mon.noname-b 2023-11-02T15:34:05.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f789e59c640 1 -- 172.21.15.174:0/3329458577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890009d60 con 0x7f78a00a9730 2023-11-02T15:34:05.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:05.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac02f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f5fac0073f0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0c5370 0x7f5fac16cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f5fac0c5370 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac166c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f5fac0cb9f0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac02ebc0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac02e870 con 0x7f5fac0c5370 2023-11-02T15:34:05.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac16d2f0 con 0x7f5fac0073f0 2023-11-02T15:34:05.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7f5fb3fff640 10 monclient(hunting): authenticate will time out at 1372.444982s 2023-11-02T15:34:05.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7fbb50700640 10 monclient: get_monmap_and_config 2023-11-02T15:34:05.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7fbb50700640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.102+0000 7fbb50700640 10 monclient: monmap: 2023-11-02T15:34:05.344 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.344 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.344 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:05.104473+0000 2023-11-02T15:34:05.344 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:05.104473+0000 2023-11-02T15:34:05.344 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.345 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.345 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:05.345 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:05.345 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:05.345 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:05.346 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:05.346 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.103+0000 7fbb50700640 1 Processor -- start 2023-11-02T15:34:05.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.103+0000 7fbb50700640 1 -- start start 2023-11-02T15:34:05.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.103+0000 7fbb50700640 10 monclient: init 2023-11-02T15:34:05.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2023-11-02T15:34:05.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb480b4570 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:05.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb480b0ed0 0x7fbb48144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.347 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-c con 0x7fbb480b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:05.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbb4813e520 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:05.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb480a9780 con 0x7fbb480b4570 2023-11-02T15:34:05.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb480aa1d0 con 0x7fbb4813e520 2023-11-02T15:34:05.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.348 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb480b4950 con 0x7fbb480b0ed0 2023-11-02T15:34:05.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.104+0000 7fbb50700640 10 monclient(hunting): authenticate will time out at 1372.447053s 2023-11-02T15:34:05.349 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.105+0000 7f789e59c640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 0x7f78a013c000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f789400c410 tx=0x7f789400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.105+0000 7f788effd640 10 client.?.objecter ms_handle_connect 0x7f78a00a9730 2023-11-02T15:34:05.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.105+0000 7f788effd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.350 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1038640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b4035990 0x7f17b40c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 10 monclient(hunting): handle_auth_done global_id 4286 payload 824 2023-11-02T15:34:05.351 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 -- 172.21.15.174:0/1498947942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b4035990 msgr2=0x7f17b40c12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b4035990 0x7f17b40c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 -- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b40c7b00 msgr2=0x7f17b40be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b40c7b00 0x7f17b40be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f17c1839640 1 -- 172.21.15.174:0/1498947942 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0094d0 con 0x7f17b402a960 2023-11-02T15:34:05.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f38d2ffd640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 0x7f38d40b1720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c610 tx=0x7f38c800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f38bbfff640 10 client.?.objecter ms_handle_connect 0x7f38d40a9730 2023-11-02T15:34:05.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f38bbfff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:58040/0 (socket says 172.21.15.174:58040) 2023-11-02T15:34:05.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 1 -- 172.21.15.174:0/3782486961 learned_addr learned my addr 172.21.15.174:0/3782486961 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 10 monclient(hunting): get_auth_request con 0x7f97e00047c0 auth_method 0 2023-11-02T15:34:05.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.356 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f97effff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.356 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.110+0000 7f214bfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f214c13e520 0x7f214c13e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.356 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f97effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f97effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f97effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f214c13e520 0x7f214c13e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:58050/0 (socket says 172.21.15.174:58050) 2023-11-02T15:34:05.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 1 -- 172.21.15.174:0/3698082269 learned_addr learned my addr 172.21.15.174:0/3698082269 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 10 monclient(hunting): get_auth_request con 0x7f214c13e520 auth_method 0 2023-11-02T15:34:05.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.111+0000 7f214bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.112+0000 7f214bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.112+0000 7f214bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.112+0000 7f214bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 10 monclient(hunting): handle_auth_done global_id 4333 payload 824 2023-11-02T15:34:05.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 -- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e80c0bc0 msgr2=0x7f97e815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e80c0bc0 0x7f97e815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 -- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e802d590 msgr2=0x7f97e80c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e802d590 0x7f97e80c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 monclient: found mon.noname-b 2023-11-02T15:34:05.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f97effff640 1 -- 172.21.15.174:0/3782486961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8033a40 con 0x7f97e00047c0 2023-11-02T15:34:05.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f17c1839640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009870 tx=0x7f17ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f17b27fc640 10 client.?.objecter ms_handle_connect 0x7f17b402a960 2023-11-02T15:34:05.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.116+0000 7f17b27fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 10 monclient(hunting): handle_auth_done global_id 4336 payload 274 2023-11-02T15:34:05.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 1 -- 172.21.15.174:0/3698082269 >> v2:172.21.15.174:3300/0 conn(0x7f214c0b0ed0 msgr2=0x7f214c144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 1 --2- 172.21.15.174:0/3698082269 >> v2:172.21.15.174:3300/0 conn(0x7f214c0b0ed0 0x7f214c144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 1 -- 172.21.15.174:0/3698082269 >> v1:172.21.15.175:6789/0 conn(0x7f214c0b4570 legacy=0x7f214c0a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 1 monclient: found mon.noname-c 2023-11-02T15:34:05.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.117+0000 7f214bfff640 1 -- 172.21.15.174:0/3698082269 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0aa4b0 con 0x7f214c13e520 2023-11-02T15:34:05.368 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.121+0000 7f65aa277640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 0x7f65a41d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004220 con 0x7f38d40a9730 2023-11-02T15:34:05.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.369 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: dump: 2023-11-02T15:34:05.370 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.370 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.370 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.370 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.370 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.371 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.371 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.371 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.371 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.371 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8004380 con 0x7f38d40a9730 2023-11-02T15:34:05.372 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 5 monclient: authenticate success, global_id 4280 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003850 con 0x7f38d40a9730 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 10 monclient: _renew_subs 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d4134010 con 0x7f38d40a9730 2023-11-02T15:34:05.373 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38d1ffb640 4 set_mon_vals no callback set 2023-11-02T15:34:05.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 10 client.4280.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 10 monclient: _renew_subs 2023-11-02T15:34:05.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d4134520 con 0x7f38d40a9730 2023-11-02T15:34:05.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f38da177640 1 librados: init done 2023-11-02T15:34:05.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 0x7f65a41d61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:39076/0 (socket says 172.21.15.174:39076) 2023-11-02T15:34:05.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 1 -- 172.21.15.174:0/222372448 learned_addr learned my addr 172.21.15.174:0/222372448 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 10 monclient(hunting): get_auth_request con 0x7f65a40b3da0 auth_method 0 2023-11-02T15:34:05.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f65aa277640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.122+0000 7f97effff640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c3f0 tx=0x7f97e400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f97ecff9640 10 client.?.objecter ms_handle_connect 0x7f97e00047c0 2023-11-02T15:34:05.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f97ecff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f38bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.378 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f38bbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:05.378 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f38bbfff640 10 monclient: dump: 2023-11-02T15:34:05.378 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.378 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.378 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.380 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.380 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac03ad30 con 0x7f17b402a960 2023-11-02T15:34:05.380 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: dump: 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.381 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.382 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.382 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.382 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.382 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.382 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.383 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.383 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac0443f0 con 0x7f17b402a960 2023-11-02T15:34:05.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 5 monclient: authenticate success, global_id 4286 2023-11-02T15:34:05.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 10 monclient: _renew_subs 2023-11-02T15:34:05.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.384 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b40bd870 con 0x7f17b402a960 2023-11-02T15:34:05.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f5fb2ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac02f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.385 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b3fff640 4 set_mon_vals no callback set 2023-11-02T15:34:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0447a0 con 0x7f17b402a960 2023-11-02T15:34:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 10 client.4286.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17b27fc640 10 monclient: dump: 2023-11-02T15:34:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.389 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 10 monclient: _renew_subs 2023-11-02T15:34:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b415b3d0 con 0x7f17b402a960 2023-11-02T15:34:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.123+0000 7f17c283b640 1 librados: init done 2023-11-02T15:34:05.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f38da177640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f38da177640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f38da177640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d404e040 con 0x7f38d40a9730 2023-11-02T15:34:05.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f65aaa78640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 0x7f65a41d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f17c283b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f17c283b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f17c283b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.124+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b415ab70 con 0x7f17b402a960 2023-11-02T15:34:05.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f65aa277640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac02f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:39090/0 (socket says 172.21.15.174:39090) 2023-11-02T15:34:05.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 1 -- 172.21.15.174:0/2909859623 learned_addr learned my addr 172.21.15.174:0/2909859623 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fac0073f0 auth_method 0 2023-11-02T15:34:05.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f5fb2ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78940035f0 con 0x7f78a00a9730 2023-11-02T15:34:05.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:05.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: dump: 2023-11-02T15:34:05.397 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.398 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.398 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.398 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.398 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.398 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.399 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f78a4a3a640 5 monclient: authenticate success, global_id 4330 2023-11-02T15:34:05.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7894003770 con 0x7f78a00a9730 2023-11-02T15:34:05.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f788effd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f78a4a3a640 10 monclient: _renew_subs 2023-11-02T15:34:05.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f78a4a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a013dea0 con 0x7f78a00a9730 2023-11-02T15:34:05.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.125+0000 7f789cd99640 4 set_mon_vals no callback set 2023-11-02T15:34:05.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 10 client.4330.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 10 monclient: _renew_subs 2023-11-02T15:34:05.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a013e380 con 0x7f78a00a9730 2023-11-02T15:34:05.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f78a4a3a640 1 librados: init done 2023-11-02T15:34:05.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894003b60 con 0x7f78a00a9730 2023-11-02T15:34:05.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f788effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f788effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f788effd640 10 monclient: dump: 2023-11-02T15:34:05.403 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.403 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.403 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.404 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.404 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.404 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.404 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.405 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.405 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.405 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7fbb4ec9d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fbb4813e520 0x7fbb4813e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:38680/0 (socket says 172.21.15.174:38680) 2023-11-02T15:34:05.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2457537144 learned_addr learned my addr 172.21.15.174:0/2457537144 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:05.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f65aaa78640 10 monclient(hunting): get_auth_request con 0x7f65a41451e0 auth_method 0 2023-11-02T15:34:05.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f65aaa78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.126+0000 7f65aaa78640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.407 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.127+0000 7f65aaa78640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.127+0000 7f78a4a3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.127+0000 7f78a4a3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.127+0000 7f78a4a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.127+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a013b8e0 con 0x7f78a00a9730 2023-11-02T15:34:05.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.128+0000 7f65a9a76640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a4141c60 0x7f65a40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.129+0000 7f5fb27fc640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0c5370 0x7f5fac16cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65a9a76640 10 monclient(hunting): get_auth_request con 0x7f65a4141c60 auth_method 0 2023-11-02T15:34:05.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65a9a76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65a9a76640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65a9a76640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65aaa78640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65aaa78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f65aaa78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request con 0x7f5fac0c5370 auth_method 0 2023-11-02T15:34:05.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f5fb27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f5fb27fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.130+0000 7f5fb27fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.131+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.132+0000 7f65a9a76640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.132+0000 7f65a9a76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.132+0000 7f65a9a76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 10 monclient(hunting): handle_auth_done global_id 4339 payload 824 2023-11-02T15:34:05.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 -- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 msgr2=0x7f65a41d61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 0x7f65a41d61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 -- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a4141c60 msgr2=0x7f65a40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a4141c60 0x7f65a40b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 monclient: found mon.noname-b 2023-11-02T15:34:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.416 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.416 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f65aaa78640 1 -- 172.21.15.174:0/222372448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00094d0 con 0x7f65a41451e0 2023-11-02T15:34:05.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request con 0x7f5fac0cb9f0 auth_method 0 2023-11-02T15:34:05.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f5fb37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f5fb37fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.133+0000 7f5fb37fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.135+0000 7f5fb27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.136+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358895916 0 0) 0x7fbb480aa1d0 con 0x7fbb4813e520 2023-11-02T15:34:05.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7fbb4d49a640 10 monclient(hunting): _negotiate 2023-11-02T15:34:05.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7fbb4d49a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7fbb4d49a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7fbb4d49a640 10 monclient(hunting): my global_id is 4342 2023-11-02T15:34:05.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb34003520 con 0x7fbb4813e520 2023-11-02T15:34:05.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7f65aaa78640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 0x7f65a41d3380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f659c00be20 tx=0x7f659c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7f659affd640 10 client.?.objecter ms_handle_connect 0x7f65a41451e0 2023-11-02T15:34:05.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.137+0000 7f659affd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 10 monclient(hunting): handle_auth_done global_id 4299 payload 824 2023-11-02T15:34:05.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 -- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 msgr2=0x7f5fac02f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac02f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 -- 172.21.15.174:0/2909859623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0c5370 msgr2=0x7f5fac16cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0c5370 0x7f5fac16cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 monclient: found mon.noname-a 2023-11-02T15:34:05.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.138+0000 7f5fb37fe640 1 -- 172.21.15.174:0/2909859623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4066970 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.150+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f17ac02f050 con 0x7f17b402a960 2023-11-02T15:34:05.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.151+0000 7f17b27fc640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 0x7f17a006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.151+0000 7f17c1038640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 0x7f17a006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.151+0000 7f17c1038640 10 monclient: get_auth_request con 0x7f17a006be10 auth_method 0 2023-11-02T15:34:05.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7f17b27fc640 10 client.4286.objecter ms_handle_connect 0x7f17a006be10 2023-11-02T15:34:05.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7f17c1038640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 0x7f17a006e2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17b4031b00 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637788286 0 0) 0x7fbb480a9780 con 0x7fbb480b4570 2023-11-02T15:34:05.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 10 monclient(hunting): _negotiate 2023-11-02T15:34:05.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 10 monclient(hunting): my global_id is 4302 2023-11-02T15:34:05.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.152+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb480aa1d0 con 0x7fbb480b4570 2023-11-02T15:34:05.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.154+0000 7f5fb37fe640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac166c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5fb404f940 tx=0x7f5fb4068a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.154+0000 7f5f9bfff640 10 client.?.objecter ms_handle_connect 0x7f5fac0cb9f0 2023-11-02T15:34:05.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.154+0000 7f5f9bfff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.165+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.0 v1:172.21.15.174:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131736809 0 0) 0x7fbb480aa1d0 con 0x7fbb480b4570 2023-11-02T15:34:05.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.165+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 >> v2:172.21.15.175:3300/0 conn(0x7fbb480b0ed0 msgr2=0x7fbb48144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.165+0000 7fbb4d49a640 1 --2- 172.21.15.174:0/2457537144 >> v2:172.21.15.175:3300/0 conn(0x7fbb480b0ed0 0x7fbb48144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 >> v1:172.21.15.175:6789/0 conn(0x7fbb4813e520 legacy=0x7fbb4813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 monclient: found mon.noname-a-legacy 2023-11-02T15:34:05.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.174:6789/0 2023-11-02T15:34:05.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> v1:172.21.15.174:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480aa4b0 con 0x7fbb480b4570 2023-11-02T15:34:05.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.0 v1:172.21.15.174:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 692219148 0 0) 0x7fbb44001670 con 0x7fbb480b4570 2023-11-02T15:34:05.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2023-11-02T15:34:05.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: dump: 2023-11-02T15:34:05.433 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.434 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.434 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.434 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.434 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.434 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.435 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.435 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.435 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.435 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] but i'm connected to v1:172.21.15.174:6789/0, reconnecting 2023-11-02T15:34:05.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 >> v1:172.21.15.174:6789/0 conn(0x7fbb480b4570 legacy=0x7fbb480a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:34:05.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34003b80 0x7fbb34003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): picked mon.a con 0x7fbb34003b80 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb340047c0 0x7fbb34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): picked mon.c con 0x7fbb340047c0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 0x7fbb3400bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): picked mon.b con 0x7fbb34009970 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480b4950 con 0x7fbb34003b80 2023-11-02T15:34:05.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3400c350 con 0x7fbb34009970 2023-11-02T15:34:05.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3400c510 con 0x7fbb340047c0 2023-11-02T15:34:05.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7fbb4d49a640 10 monclient(hunting): _finish_auth 0 2023-11-02T15:34:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40ac840 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: dump: 2023-11-02T15:34:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.443 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.443 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb40ac9a0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5fb3fff640 5 monclient: authenticate success, global_id 4299 2023-11-02T15:34:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40acd50 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.166+0000 7f5f9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5f9bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5f9bfff640 10 monclient: dump: 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.447 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _renew_subs 2023-11-02T15:34:05.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac02bdf0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb17fa640 4 set_mon_vals no callback set 2023-11-02T15:34:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 client.4299.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _renew_subs 2023-11-02T15:34:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac167390 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 1 librados: init done 2023-11-02T15:34:05.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.167+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac02ea30 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.171+0000 7fbb4e49c640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34003b80 0x7fbb34003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.172+0000 7fbb4e49c640 10 monclient(hunting): get_auth_request con 0x7fbb34003b80 auth_method 0 2023-11-02T15:34:05.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.172+0000 7fbb4e49c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.172+0000 7fbb4e49c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.172+0000 7fbb4e49c640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:34:05.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.372+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7894003cc0 con 0x7f78a00a9730 2023-11-02T15:34:05.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.372+0000 7f788effd640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 0x7f788006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.373+0000 7f789dd9b640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 0x7f788006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.373+0000 7f789dd9b640 10 monclient: get_auth_request con 0x7f788006be10 auth_method 0 2023-11-02T15:34:05.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.373+0000 7f789dd9b640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 0x7f788006e2d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78a013cbd0 tx=0x7f7890031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.373+0000 7f788effd640 10 client.4330.objecter ms_handle_connect 0x7f788006be10 2023-11-02T15:34:05.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40035f0 con 0x7f97e00047c0 2023-11-02T15:34:05.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:05.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: dump: 2023-11-02T15:34:05.455 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.455 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.455 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.455 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.455 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.456 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.456 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.456 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.456 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.456 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4003770 con 0x7f97e00047c0 2023-11-02T15:34:05.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.458 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4003b20 con 0x7f97e00047c0 2023-11-02T15:34:05.458 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 5 monclient: authenticate success, global_id 4333 2023-11-02T15:34:05.458 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 10 monclient: _renew_subs 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e8033c30 con 0x7f97e00047c0 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ee7fc640 4 set_mon_vals no callback set 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97ecff9640 10 monclient: dump: 2023-11-02T15:34:05.460 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.460 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.460 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.460 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.460 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.461 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.461 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.461 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.461 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.461 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 10 client.4333.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 10 monclient: _renew_subs 2023-11-02T15:34:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e8033380 con 0x7f97e00047c0 2023-11-02T15:34:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.376+0000 7f97f63dd640 1 librados: init done 2023-11-02T15:34:05.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.377+0000 7f97f63dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.377+0000 7f97f63dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.377+0000 7f97f63dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.377+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8005780 con 0x7f97e00047c0 2023-11-02T15:34:05.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.381+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7894057740 con 0x7f78a00a9730 2023-11-02T15:34:05.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.381+0000 7f788effd640 10 client.4330.objecter ms_dispatch 0x7f78a0000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.381+0000 7f788effd640 3 client.4330.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.381+0000 7f788effd640 3 client.4330.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.381+0000 7f788effd640 20 client.4330.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.391+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7894053a80 con 0x7f78a00a9730 2023-11-02T15:34:05.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.391+0000 7f788effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.391+0000 7f788effd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.393+0000 7f214bfff640 1 --2- 172.21.15.174:0/3698082269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c13e9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2144009e50 tx=0x7f214402bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3c030cfcec60b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.397+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 0x7fbb3400bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.398+0000 7fbb4ec9d640 10 monclient(hunting): get_auth_request con 0x7fbb34009970 auth_method 0 2023-11-02T15:34:05.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.398+0000 7fbb4ec9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.398+0000 7fbb4ec9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.398+0000 7fbb4ec9d640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T15:34:05.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.399+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.399+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.399+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-11-02T15:34:05.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00ce70 con 0x7f65a41451e0 2023-11-02T15:34:05.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:05.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: dump: 2023-11-02T15:34:05.471 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.471 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.471 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.471 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.472 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c014440 con 0x7f65a41451e0 2023-11-02T15:34:05.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 5 monclient: authenticate success, global_id 4339 2023-11-02T15:34:05.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c014830 con 0x7f65a41451e0 2023-11-02T15:34:05.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f659affd640 10 monclient: dump: 2023-11-02T15:34:05.475 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.475 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.475 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.475 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.475 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _renew_subs 2023-11-02T15:34:05.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65a8a74640 4 set_mon_vals no callback set 2023-11-02T15:34:05.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a40b1220 con 0x7f65a41451e0 2023-11-02T15:34:05.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 client.4339.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _renew_subs 2023-11-02T15:34:05.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a41d3aa0 con 0x7f65a41451e0 2023-11-02T15:34:05.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 1 librados: init done 2023-11-02T15:34:05.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a40b1440 con 0x7f65a41451e0 2023-11-02T15:34:05.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_done global_id 4302 payload 293 2023-11-02T15:34:05.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb340047c0 msgr2=0x7fbb34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb340047c0 0x7fbb34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34003b80 msgr2=0x7fbb34003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34003b80 0x7fbb34003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 monclient: found mon.b 2023-11-02T15:34:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2457537144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480aa1d0 con 0x7fbb34009970 2023-11-02T15:34:05.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb4ec9d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb50700640 5 monclient: authenticate success, global_id 4302 2023-11-02T15:34:05.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.402+0000 7fbb50700640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:05.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.403+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2457537144 reap_dead start 2023-11-02T15:34:05.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 1 -- 172.21.15.174:0/3698082269 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214402c960 con 0x7f214c13e520 2023-11-02T15:34:05.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:05.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: dump: 2023-11-02T15:34:05.486 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.486 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.487 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.487 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.487 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.487 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.487 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.488 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.488 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.488 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 1 -- 172.21.15.174:0/3698082269 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214402cac0 con 0x7f214c13e520 2023-11-02T15:34:05.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f21521c9640 5 monclient: authenticate success, global_id 4336 2023-11-02T15:34:05.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 1 -- 172.21.15.174:0/3698082269 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214402ce50 con 0x7f214c13e520 2023-11-02T15:34:05.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:05.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f214a7fc640 10 monclient: dump: 2023-11-02T15:34:05.490 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.490 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.491 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.491 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.491 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.491 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.491 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.492 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.492 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.492 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f21521c9640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:05.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.404+0000 7f21521c9640 4 set_mon_vals no callback set 2023-11-02T15:34:05.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 10 monclient: shutdown 2023-11-02T15:34:05.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 -- 172.21.15.174:0/3698082269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 msgr2=0x7f214c13e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 --2- 172.21.15.174:0/3698082269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c13e9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2144009e50 tx=0x7f214402bdc0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 -- 172.21.15.174:0/3698082269 shutdown_connections 2023-11-02T15:34:05.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 --2- 172.21.15.174:0/3698082269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c13e9b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 --2- 172.21.15.174:0/3698082269 >> v2:172.21.15.174:3300/0 conn(0x7f214c0b0ed0 0x7f214c144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 -- 172.21.15.174:0/3698082269 >> 172.21.15.174:0/3698082269 conn(0x7f214c065d50 msgr2=0x7f214c066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 -- 172.21.15.174:0/3698082269 shutdown_connections 2023-11-02T15:34:05.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.405+0000 7f21521c9640 1 -- 172.21.15.174:0/3698082269 wait complete. 2023-11-02T15:34:05.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.406+0000 7f21521c9640 10 monclient: build_initial_monmap 2023-11-02T15:34:05.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.406+0000 7f21521c9640 10 monclient: monmap: 2023-11-02T15:34:05.495 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.496 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:05.497 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:05.497 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:05.497 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.497 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 librados: starting msgr at 2023-11-02T15:34:05.497 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 librados: starting objecter 2023-11-02T15:34:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 Processor -- start 2023-11-02T15:34:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 -- start start 2023-11-02T15:34:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 librados: setting wanted keys 2023-11-02T15:34:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 1 librados: calling monclient init 2023-11-02T15:34:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.407+0000 7f21521c9640 10 monclient: init 2023-11-02T15:34:05.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:05.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:05.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f214c0b0ed0 0x7f214c144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-c con 0x7f214c0b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:05.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-a con 0x7f214c0b4570 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c1da380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): picked mon.noname-b con 0x7f214c13e520 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0aadf0 con 0x7f214c0b4570 2023-11-02T15:34:05.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c142730 con 0x7f214c13e520 2023-11-02T15:34:05.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:05.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c141d70 con 0x7f214c0b0ed0 2023-11-02T15:34:05.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:05.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.408+0000 7f21521c9640 10 monclient(hunting): authenticate will time out at 1372.751213s 2023-11-02T15:34:05.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.444+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5fb40c93e0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.444+0000 7f5f9bfff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 0x7f5fa006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.444+0000 7f5fb2ffd640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 0x7f5fa006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.444+0000 7f5fb2ffd640 10 monclient: get_auth_request con 0x7f5fa006bdc0 auth_method 0 2023-11-02T15:34:05.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.445+0000 7f5f9bfff640 10 client.4299.objecter ms_handle_connect 0x7f5fa006bdc0 2023-11-02T15:34:05.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.445+0000 7f5fb2ffd640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 0x7f5fa006e280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80094a0 tx=0x7f5fa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5fb40f4940 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 10 client.4299.objecter ms_dispatch 0x7f5fb4000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 3 client.4299.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 3 client.4299.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 20 client.4299.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5fb40f0580 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.460+0000 7f5f9bfff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.462+0000 7f214affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.464+0000 7f214affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38972/0 (socket says 172.21.15.174:38972) 2023-11-02T15:34:05.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.464+0000 7f214affd640 1 -- 172.21.15.174:0/2324108350 learned_addr learned my addr 172.21.15.174:0/2324108350 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:05.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.465+0000 7f214affd640 10 monclient(hunting): get_auth_request con 0x7f214c0b4570 auth_method 0 2023-11-02T15:34:05.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.465+0000 7f214affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:05.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.465+0000 7f214affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:05.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.465+0000 7f214affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:05.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.469+0000 7f214affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:05.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.469+0000 7f214affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:05.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.469+0000 7f214affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:05.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 10 monclient(hunting): handle_auth_done global_id 4305 payload 824 2023-11-02T15:34:05.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 -- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f214c0b0ed0 msgr2=0x7f214c144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f214c0b0ed0 0x7f214c144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 -- 172.21.15.174:0/2324108350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 msgr2=0x7f214c1da380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c1da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 10 monclient: _finish_hunting 0 2023-11-02T15:34:05.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 monclient: found mon.noname-a 2023-11-02T15:34:05.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:05.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.473+0000 7f214affd640 1 -- 172.21.15.174:0/2324108350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009b30 con 0x7f214c0b4570 2023-11-02T15:34:05.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.482+0000 7f214affd640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f214000b480 tx=0x7f214000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.482+0000 7f2133fff640 10 client.?.objecter ms_handle_connect 0x7f214c0b4570 2023-11-02T15:34:05.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.482+0000 7f2133fff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:05.559 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2140011830 con 0x7f214c0b4570 2023-11-02T15:34:05.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:05.560 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: dump: 2023-11-02T15:34:05.561 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.561 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.561 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.561 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.561 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.562 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.562 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.562 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.562 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.562 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: _finish_auth 0 2023-11-02T15:34:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21400119b0 con 0x7f214c0b4570 2023-11-02T15:34:05.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 5 monclient: authenticate success, global_id 4305 2023-11-02T15:34:05.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 10 monclient: _renew_subs 2023-11-02T15:34:05.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c1da8c0 con 0x7f214c0b4570 2023-11-02T15:34:05.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:05.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2140011d60 con 0x7f214c0b4570 2023-11-02T15:34:05.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:05.568 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2133fff640 10 monclient: dump: 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:05.569 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:05.570 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:05.570 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:05.571 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:05.571 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:05.571 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:05.571 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 10 client.4305.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:05.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 10 monclient: _renew_subs 2023-11-02T15:34:05.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f2149ffb640 4 set_mon_vals no callback set 2023-11-02T15:34:05.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c141530 con 0x7f214c0b4570 2023-11-02T15:34:05.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.556+0000 7f21521c9640 1 librados: init done 2023-11-02T15:34:05.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.557+0000 7f21521c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.557+0000 7f21521c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.557+0000 7f21521c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.557+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c141f10 con 0x7f214c0b4570 2023-11-02T15:34:05.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.581+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f214001a400 con 0x7f214c0b4570 2023-11-02T15:34:05.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.582+0000 7f2133fff640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 0x7f213806e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.585 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.582+0000 7f214b7fe640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 0x7f213806e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.585 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.582+0000 7f214b7fe640 10 monclient: get_auth_request con 0x7f213806bdc0 auth_method 0 2023-11-02T15:34:05.585 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.582+0000 7f214b7fe640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 0x7f213806e280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f213c001560 tx=0x7f213c007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.585 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.582+0000 7f2133fff640 10 client.4305.objecter ms_handle_connect 0x7f213806bdc0 2023-11-02T15:34:05.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.631+0000 7f78a4a3a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:05.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.631+0000 7f78a4a3a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:05.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.631+0000 7f78a4a3a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:05.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.631+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78a013d3a0 con 0x7f78a00a9730 2023-11-02T15:34:05.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f17ac07a990 con 0x7f17b402a960 2023-11-02T15:34:05.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 10 client.4286.objecter ms_dispatch 0x7f17bc000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 3 client.4286.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 3 client.4286.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 20 client.4286.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f17ac0765d0 con 0x7f17b402a960 2023-11-02T15:34:05.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.657+0000 7f17b27fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.674+0000 7f5fb3fff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:05.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.674+0000 7f5fb3fff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:05.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.674+0000 7f5fb3fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:05.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.674+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5fac00a370 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f38c80039b0 con 0x7f38d40a9730 2023-11-02T15:34:05.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38bbfff640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 0x7f38c006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38d37fe640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 0x7f38c006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38d37fe640 10 monclient: get_auth_request con 0x7f38c006bdc0 auth_method 0 2023-11-02T15:34:05.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38bbfff640 10 client.4280.objecter ms_handle_connect 0x7f38c006bdc0 2023-11-02T15:34:05.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.693+0000 7f38d37fe640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 0x7f38c006e280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005020 tx=0x7f38c4007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.719+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f38c8057d30 con 0x7f38d40a9730 2023-11-02T15:34:05.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.719+0000 7f38bbfff640 10 client.4280.objecter ms_dispatch 0x7f38d4000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.719+0000 7f38bbfff640 3 client.4280.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.719+0000 7f38bbfff640 3 client.4280.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.720+0000 7f38bbfff640 20 client.4280.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.720+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f38c8054070 con 0x7f38d40a9730 2023-11-02T15:34:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.720+0000 7f38bbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.720+0000 7f38bbfff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2140058a60 con 0x7f214c0b4570 2023-11-02T15:34:05.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 10 client.4305.objecter ms_dispatch 0x7f214c000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 3 client.4305.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 3 client.4305.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 20 client.4305.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f21400546a0 con 0x7f214c0b4570 2023-11-02T15:34:05.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.842+0000 7f2133fff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.856+0000 7f5f9bfff640 1 -- 172.21.15.174:0/2909859623 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fb40aaed0 con 0x7f5fac0cb9f0 2023-11-02T15:34:05.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.857+0000 7f5f9bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:05.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.857+0000 7f5f9bfff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:05.860 INFO:teuthology.orchestra.run.smithi174.stdout:60129542148 2023-11-02T15:34:05.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 10 librados: watch_flush enter 2023-11-02T15:34:05.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 10 librados: watch_flush exit 2023-11-02T15:34:05.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 20 client.4299.objecter shutdown clearing up homeless session... 2023-11-02T15:34:05.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 10 client.4299.objecter successfully canceled tick 2023-11-02T15:34:05.863 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 msgr2=0x7f5fa006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 0x7f5fa006e280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80094a0 tx=0x7f5fa8008040 comp rx=0 tx=0).stop 2023-11-02T15:34:05.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 10 monclient: shutdown 2023-11-02T15:34:05.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 msgr2=0x7f5fac166c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.859+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac166c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5fb404f940 tx=0x7f5fb4068a40 comp rx=0 tx=0).stop 2023-11-02T15:34:05.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 shutdown_connections 2023-11-02T15:34:05.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fac0cb9f0 0x7f5fac166c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0c5370 0x7f5fac16cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.865 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5fa006bdc0 0x7f5fa006e280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.866 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 --2- 172.21.15.174:0/2909859623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fac0073f0 0x7f5fac02f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.866 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 >> 172.21.15.174:0/2909859623 conn(0x7f5fac006240 msgr2=0x7f5fac0c17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.866 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 shutdown_connections 2023-11-02T15:34:05.866 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 -- 172.21.15.174:0/2909859623 wait complete. 2023-11-02T15:34:05.867 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.860+0000 7f5fb3fff640 1 librados: shutdown 2023-11-02T15:34:05.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.871+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f659c01d3b0 con 0x7f65a41451e0 2023-11-02T15:34:05.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.872+0000 7f659affd640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 0x7f658806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.872+0000 7f65aa277640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 0x7f658806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.872+0000 7f65aa277640 10 monclient: get_auth_request con 0x7f658806be10 auth_method 0 2023-11-02T15:34:05.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.872+0000 7f659affd640 10 client.4339.objecter ms_handle_connect 0x7f658806be10 2023-11-02T15:34:05.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.872+0000 7f65aa277640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 0x7f658806e2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65a002b930 tx=0x7f65a003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2023-11-02T15:34:05.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.877+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f659c0582a0 con 0x7f65a41451e0 2023-11-02T15:34:05.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.878+0000 7f659affd640 10 client.4339.objecter ms_dispatch 0x7f65a4000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.878+0000 7f659affd640 3 client.4339.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.878+0000 7f659affd640 3 client.4339.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.878+0000 7f659affd640 20 client.4339.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.887+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f659c0545e0 con 0x7f65a41451e0 2023-11-02T15:34:05.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.887+0000 7f659affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.887+0000 7f659affd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.906+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f97e4003c80 con 0x7f97e00047c0 2023-11-02T15:34:05.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.906+0000 7f97ecff9640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 0x7f97d406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:05.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.906+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 0x7f97d406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:05.910 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ef7fe640 10 monclient: get_auth_request con 0x7f97d406be40 auth_method 0 2023-11-02T15:34:05.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ef7fe640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 0x7f97d406e300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 10 client.4333.objecter ms_handle_connect 0x7f97d406be40 2023-11-02T15:34:05.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f97e4057ac0 con 0x7f97e00047c0 2023-11-02T15:34:05.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 10 client.4333.objecter ms_dispatch 0x7f97f0000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:05.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 3 client.4333.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:05.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 3 client.4333.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:05.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.907+0000 7f97ecff9640 20 client.4333.objecter dump_active .. 0 homeless 2023-11-02T15:34:05.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.919+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f97e4053700 con 0x7f97e00047c0 2023-11-02T15:34:05.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.919+0000 7f97ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:05.924 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.919+0000 7f97ecff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:05.924 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.921+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 0x7fbb3400bd60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c320 tx=0x7fbb340057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2b147739efe046 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:05.969 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.966+0000 7f38da177640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:05.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.966+0000 7f38da177640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:05.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.966+0000 7f38da177640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.966+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f38d41347f0 con 0x7f38d40a9730 2023-11-02T15:34:05.973 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.969+0000 7f17c283b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:05.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.969+0000 7f17c283b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:05.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.969+0000 7f17c283b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:05.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.969+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f17b4005780 con 0x7f17b402a960 2023-11-02T15:34:05.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.976+0000 7f38bbfff640 1 -- 172.21.15.174:0/3224807096 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38c801e070 con 0x7f38d40a9730 2023-11-02T15:34:05.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.976+0000 7f38bbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:05.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.976+0000 7f38bbfff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:05.979 INFO:teuthology.orchestra.run.smithi174.stdout:68719476739 2023-11-02T15:34:05.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 10 librados: watch_flush enter 2023-11-02T15:34:05.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 10 librados: watch_flush exit 2023-11-02T15:34:05.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 20 client.4280.objecter shutdown clearing up homeless session... 2023-11-02T15:34:05.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 10 client.4280.objecter successfully canceled tick 2023-11-02T15:34:05.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 msgr2=0x7f38c006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 0x7f38c006e280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005020 tx=0x7f38c4007ef0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 10 monclient: shutdown 2023-11-02T15:34:05.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:05.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 msgr2=0x7f38d40b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:05.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 0x7f38d40b1720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c610 tx=0x7f38c800cad0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 shutdown_connections 2023-11-02T15:34:05.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d40aeef0 0x7f38d40b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f38c006bdc0 0x7f38c006e280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38d40a9730 0x7f38d40b1720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 --2- 172.21.15.174:0/3224807096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38d406a940 0x7f38d41cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:05.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.979+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 >> 172.21.15.174:0/3224807096 conn(0x7f38d4069e50 msgr2=0x7f38d40a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:05.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.980+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 shutdown_connections 2023-11-02T15:34:05.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.980+0000 7f38da177640 1 -- 172.21.15.174:0/3224807096 wait complete. 2023-11-02T15:34:05.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.980+0000 7f38da177640 1 librados: shutdown 2023-11-02T15:34:05.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.986+0000 7f17b27fc640 1 -- 172.21.15.174:0/1498947942 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17ac079bf0 con 0x7f17b402a960 2023-11-02T15:34:05.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.986+0000 7f17b27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:05.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.986+0000 7f17b27fc640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:05.995 INFO:teuthology.orchestra.run.smithi174.stdout:60129542147 2023-11-02T15:34:05.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 10 librados: watch_flush enter 2023-11-02T15:34:05.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 10 librados: watch_flush exit 2023-11-02T15:34:05.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 20 client.4286.objecter shutdown clearing up homeless session... 2023-11-02T15:34:05.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 10 client.4286.objecter successfully canceled tick 2023-11-02T15:34:05.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 msgr2=0x7f17a006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 0x7f17a006e2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17b4031b00 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2023-11-02T15:34:06.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 10 monclient: shutdown 2023-11-02T15:34:06.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 msgr2=0x7f17b4031b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009870 tx=0x7f17ac0046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 shutdown_connections 2023-11-02T15:34:06.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b40c7b00 0x7f17b40be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b4035990 0x7f17b40c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f17a006be10 0x7f17a006e2d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 --2- 172.21.15.174:0/1498947942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b402a960 0x7f17b4031b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 >> 172.21.15.174:0/1498947942 conn(0x7f17b40064f0 msgr2=0x7f17b40295d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 shutdown_connections 2023-11-02T15:34:06.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 -- 172.21.15.174:0/1498947942 wait complete. 2023-11-02T15:34:06.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:05.990+0000 7f17c283b640 1 librados: shutdown 2023-11-02T15:34:06.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476739 for osd.2 2023-11-02T15:34:06.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4002cbf0 con 0x7fbb34009970 2023-11-02T15:34:06.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:06.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:06.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 10 monclient: dump: 2023-11-02T15:34:06.008 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:06.009 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:06.009 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.009 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.009 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:06.010 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.010 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:06.010 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:06.010 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:06.011 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4002cd50 con 0x7fbb34009970 2023-11-02T15:34:06.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb50700640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:06.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:06.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb4d49a640 1 -- 172.21.15.174:0/2457537144 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40034610 con 0x7fbb34009970 2023-11-02T15:34:06.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb50700640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:06.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7f788effd640 1 -- 172.21.15.174:0/3329458577 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78a013d3a0 con 0x7f78a00a9730 2023-11-02T15:34:06.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7f788effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:06.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7fbb50700640 4 set_mon_vals no callback set 2023-11-02T15:34:06.014 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.002+0000 7f788effd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:06.017 INFO:teuthology.orchestra.run.smithi174.stdout:68719476740 2023-11-02T15:34:06.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.003+0000 7fbb50700640 10 monclient: shutdown 2023-11-02T15:34:06.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.003+0000 7fbb4d49a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:06.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.004+0000 7fbb4d49a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:06.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.004+0000 7fbb4d49a640 10 monclient: dump: 2023-11-02T15:34:06.021 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:06.021 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:06.022 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.022 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.022 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:06.023 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.023 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:06.023 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:06.023 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:06.024 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.004+0000 7fbb50700640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.004+0000 7fbb50700640 1 -- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 msgr2=0x7fbb3400bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.004+0000 7fbb50700640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 0x7fbb3400bd60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c320 tx=0x7fbb340057f0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 10 librados: watch_flush enter 2023-11-02T15:34:06.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 10 librados: watch_flush exit 2023-11-02T15:34:06.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 20 client.4330.objecter shutdown clearing up homeless session... 2023-11-02T15:34:06.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 10 client.4330.objecter successfully canceled tick 2023-11-02T15:34:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 msgr2=0x7f788006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 0x7f788006e2d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78a013cbd0 tx=0x7f7890031000 comp rx=0 tx=0).stop 2023-11-02T15:34:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 10 monclient: shutdown 2023-11-02T15:34:06.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 msgr2=0x7f78a013c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 0x7f78a013c000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f789400c410 tx=0x7f789400c810 comp rx=0 tx=0).stop 2023-11-02T15:34:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 shutdown_connections 2023-11-02T15:34:06.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78a00aeef0 0x7f78a013d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f788006be10 0x7f788006e2d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78a00ac390 0x7f78a01369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 --2- 172.21.15.174:0/3329458577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a00a9730 0x7f78a013c000 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 >> 172.21.15.174:0/3329458577 conn(0x7f78a0069e50 msgr2=0x7f78a006a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 shutdown_connections 2023-11-02T15:34:06.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 -- 172.21.15.174:0/3329458577 wait complete. 2023-11-02T15:34:06.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7f78a4a3a640 1 librados: shutdown 2023-11-02T15:34:06.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.005+0000 7fbb50700640 1 -- 172.21.15.174:0/2457537144 shutdown_connections 2023-11-02T15:34:06.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.005+0000 7fbb50700640 1 --2- 172.21.15.174:0/2457537144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb34009970 0x7fbb3400bd60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.005+0000 7fbb50700640 1 -- 172.21.15.174:0/2457537144 >> 172.21.15.174:0/2457537144 conn(0x7fbb48065d50 msgr2=0x7fbb48066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.005+0000 7fbb50700640 1 -- 172.21.15.174:0/2457537144 shutdown_connections 2023-11-02T15:34:06.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.005+0000 7fbb50700640 1 -- 172.21.15.174:0/2457537144 wait complete. 2023-11-02T15:34:06.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.5 2023-11-02T15:34:06.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7fbb50700640 10 monclient: build_initial_monmap 2023-11-02T15:34:06.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.006+0000 7fbb50700640 10 monclient: monmap: 2023-11-02T15:34:06.038 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:06.038 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:06.038 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:06.039 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:06.039 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:06.039 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.039 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:06.039 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:06.040 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:06.040 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 librados: starting msgr at 2023-11-02T15:34:06.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 librados: starting objecter 2023-11-02T15:34:06.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 Processor -- start 2023-11-02T15:34:06.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 -- start start 2023-11-02T15:34:06.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 librados: setting wanted keys 2023-11-02T15:34:06.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 1 librados: calling monclient init 2023-11-02T15:34:06.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.007+0000 7fbb50700640 10 monclient: init 2023-11-02T15:34:06.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:06.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:34:06.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34009970 0x7fbb481d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:06.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-a con 0x7fbb34009970 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:06.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb480aafa0 0x7fbb481d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:06.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-c con 0x7fbb480aafa0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:06.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:06.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): picked mon.noname-b con 0x7fbb480abbb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:06.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:06.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48142480 con 0x7fbb34009970 2023-11-02T15:34:06.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:06.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481d69d0 con 0x7fbb480abbb0 2023-11-02T15:34:06.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:06.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480ac570 con 0x7fbb480aafa0 2023-11-02T15:34:06.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:06.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.008+0000 7fbb50700640 10 monclient(hunting): authenticate will time out at 1373.351516s 2023-11-02T15:34:06.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476740 for osd.0 2023-11-02T15:34:06.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:06.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:48792/0 (socket says 172.21.15.174:48792) 2023-11-02T15:34:06.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2071784504 learned_addr learned my addr 172.21.15.174:0/2071784504 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:06.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 10 monclient(hunting): get_auth_request con 0x7fbb480abbb0 auth_method 0 2023-11-02T15:34:06.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:06.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:06.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.021+0000 7fbb4ec9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:06.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.022+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:06.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.022+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:06.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.022+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:06.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2023-11-02T15:34:06.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb480aafa0 msgr2=0x7fbb481d73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb480aafa0 0x7fbb481d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34009970 msgr2=0x7fbb481d6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34009970 0x7fbb481d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 10 monclient: _finish_hunting 0 2023-11-02T15:34:06.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 monclient: found mon.noname-b 2023-11-02T15:34:06.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:06.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.028+0000 7fbb4ec9d640 1 -- 172.21.15.174:0/2071784504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48140470 con 0x7fbb480abbb0 2023-11-02T15:34:06.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.031+0000 7fbb4ec9d640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c440 tx=0x7fbb40001650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:06.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.031+0000 7fbb3effd640 10 client.?.objecter ms_handle_connect 0x7fbb480abbb0 2023-11-02T15:34:06.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.031+0000 7fbb3effd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:06.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.093+0000 7f65ac4db640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2023-11-02T15:34:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.093+0000 7f65ac4db640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-11-02T15:34:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.093+0000 7f65ac4db640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.093+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f65a40b09c0 con 0x7f65a41451e0 2023-11-02T15:34:06.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.102+0000 7f21521c9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2023-11-02T15:34:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.102+0000 7f21521c9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-11-02T15:34:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.102+0000 7f21521c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:06.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.102+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f214c0abaf0 con 0x7f214c0b4570 2023-11-02T15:34:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.105+0000 7f2133fff640 1 -- 172.21.15.174:0/2324108350 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2140015020 con 0x7f214c0b4570 2023-11-02T15:34:06.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.105+0000 7f2133fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2023-11-02T15:34:06.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.105+0000 7f2133fff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:06.108 INFO:teuthology.orchestra.run.smithi174.stdout:68719476742 2023-11-02T15:34:06.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 10 librados: watch_flush enter 2023-11-02T15:34:06.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 10 librados: watch_flush exit 2023-11-02T15:34:06.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 20 client.4305.objecter shutdown clearing up homeless session... 2023-11-02T15:34:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 10 client.4305.objecter successfully canceled tick 2023-11-02T15:34:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 msgr2=0x7f213806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 0x7f213806e280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f213c001560 tx=0x7f213c007990 comp rx=0 tx=0).stop 2023-11-02T15:34:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 10 monclient: shutdown 2023-11-02T15:34:06.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 msgr2=0x7f214c1421f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f214000b480 tx=0x7f214000b940 comp rx=0 tx=0).stop 2023-11-02T15:34:06.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 shutdown_connections 2023-11-02T15:34:06.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f214c13e520 0x7f214c1da380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f214c0b4570 0x7f214c1421f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f213806bdc0 0x7f213806e280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 --2- 172.21.15.174:0/2324108350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f214c0b0ed0 0x7f214c144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 >> 172.21.15.174:0/2324108350 conn(0x7f214c065d50 msgr2=0x7f214c0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 shutdown_connections 2023-11-02T15:34:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 -- 172.21.15.174:0/2324108350 wait complete. 2023-11-02T15:34:06.114 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.107+0000 7f21521c9640 1 librados: shutdown 2023-11-02T15:34:06.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476742 for osd.1 2023-11-02T15:34:06.124 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:06.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.145+0000 7f97f63dd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2023-11-02T15:34:06.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.145+0000 7f97f63dd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-11-02T15:34:06.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.145+0000 7f97f63dd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.149 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.145+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f97e8034030 con 0x7f97e00047c0 2023-11-02T15:34:06.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.327+0000 7f97ecff9640 1 -- 172.21.15.174:0/3782486961 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97e4017020 con 0x7f97e00047c0 2023-11-02T15:34:06.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.327+0000 7f97ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2023-11-02T15:34:06.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.327+0000 7f97ecff9640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:06.330 INFO:teuthology.orchestra.run.smithi174.stdout:60129542150 2023-11-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 10 librados: watch_flush enter 2023-11-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 10 librados: watch_flush exit 2023-11-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 20 client.4333.objecter shutdown clearing up homeless session... 2023-11-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 10 client.4333.objecter successfully canceled tick 2023-11-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 msgr2=0x7f97d406e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 0x7f97d406e300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02f000 comp rx=0 tx=0).stop 2023-11-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 10 monclient: shutdown 2023-11-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 msgr2=0x7f97e80c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c3f0 tx=0x7f97e400c8b0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 shutdown_connections 2023-11-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f97d406be40 0x7f97d406e300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97e80c0bc0 0x7f97e815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e802d590 0x7f97e80c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 --2- 172.21.15.174:0/3782486961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97e00047c0 0x7f97e80c73b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 >> 172.21.15.174:0/3782486961 conn(0x7f97e8007090 msgr2=0x7f97e8037ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 shutdown_connections 2023-11-02T15:34:06.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 -- 172.21.15.174:0/3782486961 wait complete. 2023-11-02T15:34:06.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.329+0000 7f97f63dd640 1 librados: shutdown 2023-11-02T15:34:06.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40039070 con 0x7fbb480abbb0 2023-11-02T15:34:06.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:06.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:06.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: dump: 2023-11-02T15:34:06.339 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:06.339 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:06.339 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.339 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.340 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:06.340 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.340 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:06.340 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:06.341 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:06.341 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: _finish_auth 0 2023-11-02T15:34:06.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:06.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:06.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb400019c0 con 0x7fbb480abbb0 2023-11-02T15:34:06.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 5 monclient: authenticate success, global_id 4351 2023-11-02T15:34:06.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 10 monclient: _renew_subs 2023-11-02T15:34:06.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.342 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb48140660 con 0x7fbb480abbb0 2023-11-02T15:34:06.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:06.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40034b90 con 0x7fbb480abbb0 2023-11-02T15:34:06.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:06.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:06.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3effd640 10 monclient: dump: 2023-11-02T15:34:06.346 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:06.347 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb3ffff640 4 set_mon_vals no callback set 2023-11-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 10 client.4351.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 10 monclient: _renew_subs 2023-11-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481d9f50 con 0x7fbb480abbb0 2023-11-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.333+0000 7fbb50700640 1 librados: init done 2023-11-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.334+0000 7fbb50700640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.334+0000 7fbb50700640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.335+0000 7fbb50700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.335+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48067920 con 0x7fbb480abbb0 2023-11-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.336+0000 7f659affd640 1 -- 172.21.15.174:0/222372448 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f659c019090 con 0x7f65a41451e0 2023-11-02T15:34:06.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.336+0000 7f659affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2023-11-02T15:34:06.355 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.336+0000 7f659affd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:06.356 INFO:teuthology.orchestra.run.smithi174.stdout:60129542150 2023-11-02T15:34:06.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 10 librados: watch_flush enter 2023-11-02T15:34:06.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 10 librados: watch_flush exit 2023-11-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 20 client.4339.objecter shutdown clearing up homeless session... 2023-11-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 10 client.4339.objecter successfully canceled tick 2023-11-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 msgr2=0x7f658806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 0x7f658806e2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65a002b930 tx=0x7f65a003c040 comp rx=0 tx=0).stop 2023-11-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 10 monclient: shutdown 2023-11-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 msgr2=0x7f65a41d3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 0x7f65a41d3380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f659c00be20 tx=0x7f659c00bf10 comp rx=0 tx=0).stop 2023-11-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 shutdown_connections 2023-11-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a41451e0 0x7f65a41d3380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a4141c60 0x7f65a40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f658806be10 0x7f658806e2d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 --2- 172.21.15.174:0/222372448 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a40b3da0 0x7f65a41d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 >> 172.21.15.174:0/222372448 conn(0x7f65a4065960 msgr2=0x7f65a40b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 shutdown_connections 2023-11-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 -- 172.21.15.174:0/222372448 wait complete. 2023-11-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.339+0000 7f65ac4db640 1 librados: shutdown 2023-11-02T15:34:06.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2023-11-02T15:34:06.369 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:06.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-11-02T15:34:06.370 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:06.582 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.580+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb40034cf0 con 0x7fbb480abbb0 2023-11-02T15:34:06.582 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.580+0000 7fbb3effd640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 0x7fbb2c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:06.582 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.580+0000 7fbb4e49c640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 0x7fbb2c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:06.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.580+0000 7fbb4e49c640 10 monclient: get_auth_request con 0x7fbb2c06be10 auth_method 0 2023-11-02T15:34:06.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.581+0000 7fbb3effd640 10 client.4351.objecter ms_handle_connect 0x7fbb2c06be10 2023-11-02T15:34:06.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.581+0000 7fbb4e49c640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 0x7fbb2c06e2d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb440077e0 tx=0x7fbb44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.584+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbb40002090 con 0x7fbb480abbb0 2023-11-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.584+0000 7fbb3effd640 10 client.4351.objecter ms_dispatch 0x7fbb48000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.584+0000 7fbb3effd640 3 client.4351.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:06.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.584+0000 7fbb3effd640 3 client.4351.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:06.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.584+0000 7fbb3effd640 20 client.4351.objecter dump_active .. 0 homeless 2023-11-02T15:34:06.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.595+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbb400753b0 con 0x7fbb480abbb0 2023-11-02T15:34:06.597 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.595+0000 7fbb3effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:06.597 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.595+0000 7fbb3effd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:06.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.815+0000 7fbb50700640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:06.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.815+0000 7fbb50700640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:06.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.815+0000 7fbb50700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.815+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbb48144110 con 0x7fbb480abbb0 2023-11-02T15:34:06.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.820+0000 7fbb3effd640 1 -- 172.21.15.174:0/2071784504 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb40042020 con 0x7fbb480abbb0 2023-11-02T15:34:06.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.820+0000 7fbb3effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:06.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.820+0000 7fbb3effd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:06.822 INFO:teuthology.orchestra.run.smithi174.stdout:55834574852 2023-11-02T15:34:06.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.821+0000 7fbb50700640 10 librados: watch_flush enter 2023-11-02T15:34:06.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.821+0000 7fbb50700640 10 librados: watch_flush exit 2023-11-02T15:34:06.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.821+0000 7fbb50700640 20 client.4351.objecter shutdown clearing up homeless session... 2023-11-02T15:34:06.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.821+0000 7fbb50700640 10 client.4351.objecter successfully canceled tick 2023-11-02T15:34:06.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 msgr2=0x7fbb2c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 0x7fbb2c06e2d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb440077e0 tx=0x7fbb44009040 comp rx=0 tx=0).stop 2023-11-02T15:34:06.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 10 monclient: shutdown 2023-11-02T15:34:06.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:06.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 msgr2=0x7fbb480ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:06.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c440 tx=0x7fbb40001650 comp rx=0 tx=0).stop 2023-11-02T15:34:06.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 shutdown_connections 2023-11-02T15:34:06.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb480abbb0 0x7fbb480ac030 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fbb2c06be10 0x7fbb2c06e2d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.826 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb480aafa0 0x7fbb481d73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 --2- 172.21.15.174:0/2071784504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb34009970 0x7fbb481d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:06.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 >> 172.21.15.174:0/2071784504 conn(0x7fbb48065d50 msgr2=0x7fbb480a8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:06.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 shutdown_connections 2023-11-02T15:34:06.827 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 -- 172.21.15.174:0/2071784504 wait complete. 2023-11-02T15:34:06.828 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.822+0000 7fbb50700640 1 librados: shutdown 2023-11-02T15:34:06.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-11-02T15:34:06.877 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-02T15:34:06.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.982+0000 7fb79232e640 10 monclient: get_monmap_and_config 2023-11-02T15:34:06.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.982+0000 7fb79232e640 10 monclient: build_initial_monmap 2023-11-02T15:34:06.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.982+0000 7fb79232e640 10 monclient: monmap: 2023-11-02T15:34:06.986 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:06.987 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:06.987 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:06.983437+0000 2023-11-02T15:34:06.987 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:06.983437+0000 2023-11-02T15:34:06.987 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:06.988 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:06.988 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:06.988 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:06.988 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:06.988 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:06.989 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:06.989 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:06.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 Processor -- start 2023-11-02T15:34:06.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 -- start start 2023-11-02T15:34:06.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient: init 2023-11-02T15:34:06.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:06.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient: _add_conns ranks=[0,3,1,2,4] 2023-11-02T15:34:06.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb78c0b3850 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:06.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb78c0b3c30 0x7fb78c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:06.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-a con 0x7fb78c0b3c30 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:06.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb78c0b0950 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:06.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c13f570 con 0x7fb78c0b3850 2023-11-02T15:34:06.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c0b1470 con 0x7fb78c0b0950 2023-11-02T15:34:06.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:06.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78c141900 con 0x7fb78c0b3c30 2023-11-02T15:34:06.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:06.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.983+0000 7fb79232e640 10 monclient(hunting): authenticate will time out at 1374.325579s 2023-11-02T15:34:07.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.984+0000 7fb78b7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb78c0b3c30 0x7fb78c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.984+0000 7fb78bfff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb78c0b3850 0x7fb78c13ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:51366/0 (socket says 172.21.15.174:51366) 2023-11-02T15:34:07.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.984+0000 7fb78bfff640 1 -- 172.21.15.174:0/3008082871 learned_addr learned my addr 172.21.15.174:0/3008082871 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:07.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.985+0000 7fb78b7fe640 10 monclient(hunting): get_auth_request con 0x7fb78c0b3c30 auth_method 0 2023-11-02T15:34:07.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.985+0000 7fb78b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.985+0000 7fb78b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.985+0000 7fb78b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.986+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.986+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.986+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586267030 0 0) 0x7fb78c0b1470 con 0x7fb78c0b0950 2023-11-02T15:34:07.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 10 monclient(hunting): _negotiate 2023-11-02T15:34:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 10 monclient(hunting): my global_id is 4354 2023-11-02T15:34:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.987+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb774003520 con 0x7fb78c0b0950 2023-11-02T15:34:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042596080 0 0) 0x7fb78c13f570 con 0x7fb78c0b3850 2023-11-02T15:34:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 10 monclient(hunting): _negotiate 2023-11-02T15:34:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 10 monclient(hunting): my global_id is 4311 2023-11-02T15:34:07.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.988+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb78c0b1470 con 0x7fb78c0b3850 2023-11-02T15:34:07.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_done global_id 4308 payload 274 2023-11-02T15:34:07.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3008082871 >> v1:172.21.15.175:6789/0 conn(0x7fb78c0b0950 legacy=0x7fb78c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3008082871 >> v1:172.21.15.174:6789/0 conn(0x7fb78c0b3850 legacy=0x7fb78c13ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 1 monclient: found mon.noname-a 2023-11-02T15:34:07.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.014 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.989+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3008082871 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c142800 con 0x7fb78c0b3c30 2023-11-02T15:34:07.014 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.992+0000 7fb78b7fe640 1 --2- 172.21.15.174:0/3008082871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3c30 0x7fb78c0ae8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0097a0 tx=0x7fb77c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347f3817c67b3816 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.014 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-02T15:34:07.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c031830 con 0x7fb78c0b3c30 2023-11-02T15:34:07.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:07.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: dump: 2023-11-02T15:34:07.021 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.022 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.022 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.022 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.022 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.022 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.023 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.023 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.023 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.023 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c031990 con 0x7fb78c0b3c30 2023-11-02T15:34:07.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 1 -- 172.21.15.174:0/3008082871 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c031d20 con 0x7fb78c0b3c30 2023-11-02T15:34:07.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb79232e640 5 monclient: authenticate success, global_id 4308 2023-11-02T15:34:07.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb79232e640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:07.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb79232e640 4 set_mon_vals no callback set 2023-11-02T15:34:07.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:07.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.996+0000 7fb78affd640 10 monclient: dump: 2023-11-02T15:34:07.026 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.026 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.026 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.027 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.027 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.027 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.027 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.027 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.028 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.028 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 10 monclient: shutdown 2023-11-02T15:34:07.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 -- 172.21.15.174:0/3008082871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3c30 msgr2=0x7fb78c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 --2- 172.21.15.174:0/3008082871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3c30 0x7fb78c0ae8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0097a0 tx=0x7fb77c02b8d0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 -- 172.21.15.174:0/3008082871 shutdown_connections 2023-11-02T15:34:07.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 --2- 172.21.15.174:0/3008082871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3c30 0x7fb78c0ae8e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 -- 172.21.15.174:0/3008082871 >> 172.21.15.174:0/3008082871 conn(0x7fb78c065960 msgr2=0x7fb78c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 -- 172.21.15.174:0/3008082871 shutdown_connections 2023-11-02T15:34:07.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.997+0000 7fb79232e640 1 -- 172.21.15.174:0/3008082871 wait complete. 2023-11-02T15:34:07.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.998+0000 7fb79232e640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.998+0000 7fb79232e640 10 monclient: monmap: 2023-11-02T15:34:07.031 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.031 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:07.031 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:07.031 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:07.031 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 librados: starting msgr at 2023-11-02T15:34:07.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 librados: starting objecter 2023-11-02T15:34:07.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 Processor -- start 2023-11-02T15:34:07.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 -- start start 2023-11-02T15:34:07.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 librados: setting wanted keys 2023-11-02T15:34:07.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 1 librados: calling monclient init 2023-11-02T15:34:07.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 10 monclient: init 2023-11-02T15:34:07.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:06.999+0000 7fb79232e640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:07.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb78c0b0950 0x7fb78c1d9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-b con 0x7fb78c0b0950 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:07.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-a con 0x7fb78c0b3850 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:07.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb78c0b3c30 0x7fb78c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): picked mon.noname-c con 0x7fb78c0b3c30 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c13afc0 con 0x7fb78c0b3850 2023-11-02T15:34:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c145c00 con 0x7fb78c0b0950 2023-11-02T15:34:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1da340 con 0x7fb78c0b3c30 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb79232e640 10 monclient(hunting): authenticate will time out at 1374.341743s 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb7908cb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb78c0b3c30 0x7fb78c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb78c0b0950 0x7fb78c1d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39472/0 (socket says 172.21.15.174:39472) 2023-11-02T15:34:07.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3430037646 learned_addr learned my addr 172.21.15.174:0/3430037646 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 10 monclient(hunting): get_auth_request con 0x7fb78c0b3850 auth_method 0 2023-11-02T15:34:07.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb78b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb7908cb640 10 monclient(hunting): get_auth_request con 0x7fb78c0b3c30 auth_method 0 2023-11-02T15:34:07.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb7908cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb7908cb640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.000+0000 7fb7908cb640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): get_auth_request con 0x7fb78c0b0950 auth_method 0 2023-11-02T15:34:07.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb78bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb7908cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb7908cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.001+0000 7fb7908cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 10 monclient(hunting): handle_auth_done global_id 4314 payload 824 2023-11-02T15:34:07.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb78c0b3c30 msgr2=0x7fb78c13b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb78c0b3c30 0x7fb78c13b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3430037646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb78c0b0950 msgr2=0x7fb78c1d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb78c0b0950 0x7fb78c1d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 monclient: found mon.noname-a 2023-11-02T15:34:07.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.004+0000 7fb78b7fe640 1 -- 172.21.15.174:0/3430037646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c009400 con 0x7fb78c0b3850 2023-11-02T15:34:07.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.006+0000 7fb78b7fe640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb77c034040 tx=0x7fb77c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.006+0000 7fb788ff9640 10 client.?.objecter ms_handle_connect 0x7fb78c0b3850 2023-11-02T15:34:07.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.006+0000 7fb788ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:07.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c03a830 con 0x7fb78c0b3850 2023-11-02T15:34:07.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:07.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: dump: 2023-11-02T15:34:07.047 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.047 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.047 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.047 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.048 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.048 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.048 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.048 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.048 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.049 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c03a9b0 con 0x7fb78c0b3850 2023-11-02T15:34:07.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c03ad60 con 0x7fb78c0b3850 2023-11-02T15:34:07.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 5 monclient: authenticate success, global_id 4314 2023-11-02T15:34:07.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 10 monclient: _renew_subs 2023-11-02T15:34:07.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1da560 con 0x7fb78c0b3850 2023-11-02T15:34:07.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:07.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb78a7fc640 4 set_mon_vals no callback set 2023-11-02T15:34:07.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb788ff9640 10 monclient: dump: 2023-11-02T15:34:07.052 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.052 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.053 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.054 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.054 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 10 client.4314.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:07.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 10 monclient: _renew_subs 2023-11-02T15:34:07.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1433f0 con 0x7fb78c0b3850 2023-11-02T15:34:07.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.010+0000 7fb79232e640 1 librados: init done 2023-11-02T15:34:07.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.011+0000 7fb79232e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.011+0000 7fb79232e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.011+0000 7fb79232e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.011+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c04e040 con 0x7fb78c0b3850 2023-11-02T15:34:07.062 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-02T15:34:07.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-02T15:34:07.064 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.029+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb77c04d440 con 0x7fb78c0b3850 2023-11-02T15:34:07.065 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.029+0000 7fb788ff9640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 0x7fb77806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.065 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.029+0000 7fb78bfff640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 0x7fb77806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.065 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.029+0000 7fb78bfff640 10 monclient: get_auth_request con 0x7fb77806be10 auth_method 0 2023-11-02T15:34:07.065 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.030+0000 7fb788ff9640 10 client.4314.objecter ms_handle_connect 0x7fb77806be10 2023-11-02T15:34:07.065 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.030+0000 7fb78bfff640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 0x7fb77806e2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7800011e0 tx=0x7fb780007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.066 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.031+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb77c07a1b0 con 0x7fb78c0b3850 2023-11-02T15:34:07.066 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.031+0000 7fb788ff9640 10 client.4314.objecter ms_dispatch 0x7fb78c000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:07.066 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.031+0000 7fb788ff9640 3 client.4314.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:07.066 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.031+0000 7fb788ff9640 3 client.4314.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:07.067 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.031+0000 7fb788ff9640 20 client.4314.objecter dump_active .. 0 homeless 2023-11-02T15:34:07.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f54c4df0640 10 monclient: get_monmap_and_config 2023-11-02T15:34:07.168 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f54c4df0640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f10d1eb4640 10 monclient: get_monmap_and_config 2023-11-02T15:34:07.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f10d1eb4640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f10d1eb4640 10 monclient: monmap: 2023-11-02T15:34:07.170 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.170 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.170 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:07.167725+0000 2023-11-02T15:34:07.171 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:07.167725+0000 2023-11-02T15:34:07.171 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.171 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.171 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:07.171 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:07.172 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:07.172 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:07.172 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:07.172 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.166+0000 7f54c4df0640 10 monclient: monmap: 2023-11-02T15:34:07.173 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.173 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.173 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:07.167724+0000 2023-11-02T15:34:07.174 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:07.167724+0000 2023-11-02T15:34:07.174 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.174 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.174 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:07.174 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:07.175 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:07.175 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:07.175 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:07.175 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f8c838d5640 10 monclient: get_monmap_and_config 2023-11-02T15:34:07.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f8c838d5640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f8c838d5640 10 monclient: monmap: 2023-11-02T15:34:07.177 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.177 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.177 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:07.168097+0000 2023-11-02T15:34:07.177 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:07.168097+0000 2023-11-02T15:34:07.178 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.178 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.178 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:07.178 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:07.178 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:07.179 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:07.179 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:07.179 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f54c4df0640 1 Processor -- start 2023-11-02T15:34:07.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f54c4df0640 1 -- start start 2023-11-02T15:34:07.180 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f54c4df0640 10 monclient: init 2023-11-02T15:34:07.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f10d1eb4640 1 Processor -- start 2023-11-02T15:34:07.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f10d1eb4640 1 -- start start 2023-11-02T15:34:07.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f10d1eb4640 10 monclient: init 2023-11-02T15:34:07.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.167+0000 7f8c838d5640 1 Processor -- start 2023-11-02T15:34:07.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 -- start start 2023-11-02T15:34:07.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient: init 2023-11-02T15:34:07.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient: _add_conns ranks=[4,2,1,3,0] 2023-11-02T15:34:07.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2023-11-02T15:34:07.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10cc06b7d0 0x7f10cc0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-c con 0x7f10cc06b7d0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:07.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f54c00b4570 0x7f54c0144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-a con 0x7f54c00b4570 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:07.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54c00b0ed0 0x7f54c00a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-c con 0x7f54c00b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:07.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b1cb0 0x7f54c00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0adff0 0x7f10cc144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-b con 0x7f10cc0adff0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-b con 0x7f54c00b1cb0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c00b4950 con 0x7f54c00b4570 2023-11-02T15:34:07.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c00aa1d0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00aa4b0 con 0x7f54c00b1cb0 2023-11-02T15:34:07.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f54c4df0640 10 monclient(hunting): authenticate will time out at 1374.510044s 2023-11-02T15:34:07.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f10cc0ae460 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:07.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10cc06bbb0 con 0x7f10cc0ae460 2023-11-02T15:34:07.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10cc0af5b0 con 0x7f10cc06b7d0 2023-11-02T15:34:07.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc0ad0d0 con 0x7f10cc0adff0 2023-11-02T15:34:07.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f10d1eb4640 10 monclient(hunting): authenticate will time out at 1374.510189s 2023-11-02T15:34:07.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2023-11-02T15:34:07.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8c7c0b4570 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:07.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8c7c0b0ed0 0x7f8c7c144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-c con 0x7f8c7c0b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:07.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-b con 0x7f8c7c13e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c0a9780 con 0x7f8c7c0b4570 2023-11-02T15:34:07.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b4950 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0aa1d0 con 0x7f8c7c13e520 2023-11-02T15:34:07.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.168+0000 7f8c838d5640 10 monclient(hunting): authenticate will time out at 1374.510607s 2023-11-02T15:34:07.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54c00b0ed0 0x7f54c00a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bed9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b1cb0 0x7f54c00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f10cb7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10cc06b7d0 0x7f10cc0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54c00b0ed0 0x7f54c00a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:48830/0 (socket says 172.21.15.174:48830) 2023-11-02T15:34:07.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 1 -- 172.21.15.174:0/3062392159 learned_addr learned my addr 172.21.15.174:0/3062392159 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 10 monclient(hunting): get_auth_request con 0x7f54c00b0ed0 auth_method 0 2023-11-02T15:34:07.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bdd9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bed9d640 10 monclient(hunting): get_auth_request con 0x7f54c00b1cb0 auth_method 0 2023-11-02T15:34:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bed9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.169+0000 7f54bed9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f10cc06b7d0 0x7f10cc0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:48822/0 (socket says 172.21.15.174:48822) 2023-11-02T15:34:07.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 1 -- 172.21.15.174:0/221512757 learned_addr learned my addr 172.21.15.174:0/221512757 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 10 monclient(hunting): get_auth_request con 0x7f10cc06b7d0 auth_method 0 2023-11-02T15:34:07.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10cb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:58054/0 (socket says 172.21.15.174:58054) 2023-11-02T15:34:07.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 1 -- 172.21.15.174:0/2579296184 learned_addr learned my addr 172.21.15.174:0/2579296184 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 10 monclient(hunting): get_auth_request con 0x7f8c7c13e520 auth_method 0 2023-11-02T15:34:07.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f8c81e72640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.170+0000 7f10caffd640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0adff0 0x7f10cc144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f8c80e70640 1 --2- 172.21.15.174:0/2579296184 >> v2:172.21.15.175:3300/0 conn(0x7f8c7c0b0ed0 0x7f8c7c144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f8c81e72640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f8c81e72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.171+0000 7f8c81e72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 10 monclient(hunting): handle_auth_done global_id 4360 payload 274 2023-11-02T15:34:07.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 -- 172.21.15.174:0/3062392159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b1cb0 msgr2=0x7f54c00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 --2- 172.21.15.174:0/3062392159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b1cb0 0x7f54c00b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 -- 172.21.15.174:0/3062392159 >> v2:172.21.15.174:3300/0 conn(0x7f54c00b4570 msgr2=0x7f54c0144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 --2- 172.21.15.174:0/3062392159 >> v2:172.21.15.174:3300/0 conn(0x7f54c00b4570 0x7f54c0144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 monclient: found mon.noname-c 2023-11-02T15:34:07.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.172+0000 7f54bdd9b640 1 -- 172.21.15.174:0/3062392159 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00a8b50 con 0x7f54c00b0ed0 2023-11-02T15:34:07.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_done global_id 4363 payload 274 2023-11-02T15:34:07.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 1 -- 172.21.15.174:0/221512757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0adff0 msgr2=0x7f10cc144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0adff0 0x7f10cc144eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 1 -- 172.21.15.174:0/221512757 >> v1:172.21.15.175:6789/0 conn(0x7f10cc0ae460 legacy=0x7f10cc1453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 1 monclient: found mon.noname-c 2023-11-02T15:34:07.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f10cb7fe640 1 -- 172.21.15.174:0/221512757 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc0ad3b0 con 0x7f10cc06b7d0 2023-11-02T15:34:07.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f8c80e70640 10 monclient(hunting): get_auth_request con 0x7f8c7c0b0ed0 auth_method 0 2023-11-02T15:34:07.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f8c80e70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f8c80e70640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.174+0000 7f8c80e70640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 10 monclient(hunting): handle_auth_done global_id 4304 payload 274 2023-11-02T15:34:07.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 1 -- 172.21.15.174:0/2579296184 >> v2:172.21.15.175:3300/0 conn(0x7f8c7c0b0ed0 msgr2=0x7f8c7c144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 1 --2- 172.21.15.174:0/2579296184 >> v2:172.21.15.175:3300/0 conn(0x7f8c7c0b0ed0 0x7f8c7c144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 1 -- 172.21.15.174:0/2579296184 >> v1:172.21.15.175:6789/0 conn(0x7f8c7c0b4570 legacy=0x7f8c7c0a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 1 monclient: found mon.noname-b 2023-11-02T15:34:07.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.175+0000 7f8c81e72640 1 -- 172.21.15.174:0/2579296184 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0aa4b0 con 0x7f8c7c13e520 2023-11-02T15:34:07.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.178+0000 7f54bdd9b640 1 --2- 172.21.15.174:0/3062392159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00a9070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009850 tx=0x7f54b402b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a22b7f1122b4430 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.179+0000 7f8c81e72640 1 --2- 172.21.15.174:0/2579296184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e80 tx=0x7f8c7402bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f096bc6cae1649b server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f10cb7fe640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc0adab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009850 tx=0x7f10bc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2937c133722ee91 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 1 -- 172.21.15.174:0/2579296184 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7402c930 con 0x7f8c7c13e520 2023-11-02T15:34:07.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: dump: 2023-11-02T15:34:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.224 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.224 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.224 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.224 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 1 -- 172.21.15.174:0/2579296184 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7402ca90 con 0x7f8c7c13e520 2023-11-02T15:34:07.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c838d5640 5 monclient: authenticate success, global_id 4304 2023-11-02T15:34:07.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c838d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:07.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c838d5640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:07.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 1 -- 172.21.15.174:0/2579296184 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7402ce20 con 0x7f8c7c13e520 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c838d5640 4 set_mon_vals no callback set 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.183+0000 7f8c73fff640 10 monclient: dump: 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.227 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.228 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.229 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.229 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.229 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.229 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.229 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.230 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.230 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 10 monclient: shutdown 2023-11-02T15:34:07.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 -- 172.21.15.174:0/2579296184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 msgr2=0x7f8c7c13e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 --2- 172.21.15.174:0/2579296184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e80 tx=0x7f8c7402bd90 comp rx=0 tx=0).stop 2023-11-02T15:34:07.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 -- 172.21.15.174:0/2579296184 shutdown_connections 2023-11-02T15:34:07.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 --2- 172.21.15.174:0/2579296184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c13e520 0x7f8c7c13e9b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 --2- 172.21.15.174:0/2579296184 >> v2:172.21.15.175:3300/0 conn(0x7f8c7c0b0ed0 0x7f8c7c144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 -- 172.21.15.174:0/2579296184 >> 172.21.15.174:0/2579296184 conn(0x7f8c7c065d50 msgr2=0x7f8c7c066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 -- 172.21.15.174:0/2579296184 shutdown_connections 2023-11-02T15:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 1 -- 172.21.15.174:0/2579296184 wait complete. 2023-11-02T15:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.184+0000 7f8c838d5640 10 monclient: monmap: 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.233 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.234 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:07.234 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:07.234 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:07.234 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 librados: starting msgr at 2023-11-02T15:34:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 librados: starting objecter 2023-11-02T15:34:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 Processor -- start 2023-11-02T15:34:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 -- start start 2023-11-02T15:34:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 librados: setting wanted keys 2023-11-02T15:34:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 librados: calling monclient init 2023-11-02T15:34:07.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient: init 2023-11-02T15:34:07.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:07.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-c con 0x7f8c7c0b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4570 0x7f8c7c1412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-a con 0x7f8c7c0b4570 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:07.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7c13e520 0x7f8c7c1364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): picked mon.noname-b con 0x7f8c7c13e520 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:07.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b7b70 con 0x7f8c7c0b4570 2023-11-02T15:34:07.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c141830 con 0x7f8c7c13e520 2023-11-02T15:34:07.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c141aa0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.185+0000 7f8c838d5640 10 monclient(hunting): authenticate will time out at 1374.527646s 2023-11-02T15:34:07.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 1 -- 172.21.15.174:0/3062392159 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4004230 con 0x7f54c00b0ed0 2023-11-02T15:34:07.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:07.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: dump: 2023-11-02T15:34:07.241 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.241 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.242 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.242 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.242 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.242 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.242 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.243 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.243 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.243 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 1 -- 172.21.15.174:0/3062392159 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b402f050 con 0x7f54c00b0ed0 2023-11-02T15:34:07.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 1 -- 172.21.15.174:0/3062392159 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003610 con 0x7f54c00b0ed0 2023-11-02T15:34:07.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54c4df0640 5 monclient: authenticate success, global_id 4360 2023-11-02T15:34:07.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54c4df0640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:07.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54c4df0640 4 set_mon_vals no callback set 2023-11-02T15:34:07.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:07.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f54bd59a640 10 monclient: dump: 2023-11-02T15:34:07.246 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.246 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.246 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.246 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.247 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.247 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.247 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.247 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.247 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.248 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:58066/0 (socket says 172.21.15.174:58066) 2023-11-02T15:34:07.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 1 -- 172.21.15.174:0/3991103625 learned_addr learned my addr 172.21.15.174:0/3991103625 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 10 monclient(hunting): get_auth_request con 0x7f8c7c0b0ed0 auth_method 0 2023-11-02T15:34:07.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.186+0000 7f8c81671640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 10 monclient: shutdown 2023-11-02T15:34:07.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 -- 172.21.15.174:0/3062392159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 msgr2=0x7f54c00a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3062392159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00a9070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009850 tx=0x7f54b402b7b0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f8c81671640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f8c81671640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f8c81671640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 -- 172.21.15.174:0/3062392159 shutdown_connections 2023-11-02T15:34:07.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3062392159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b1cb0 0x7f54c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3062392159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00a9070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3062392159 >> v2:172.21.15.174:3300/0 conn(0x7f54c00b4570 0x7f54c0144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 -- 172.21.15.174:0/3062392159 >> 172.21.15.174:0/3062392159 conn(0x7f54c0065d50 msgr2=0x7f54c0066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 -- 172.21.15.174:0/3062392159 shutdown_connections 2023-11-02T15:34:07.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.187+0000 7f54c4df0640 1 -- 172.21.15.174:0/3062392159 wait complete. 2023-11-02T15:34:07.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.188+0000 7f54c4df0640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.188+0000 7f54c4df0640 10 monclient: monmap: 2023-11-02T15:34:07.255 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.255 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:07.256 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:07.256 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:07.256 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.256 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.256 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:07.257 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:07.257 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:07.257 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 librados: starting msgr at 2023-11-02T15:34:07.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 librados: starting objecter 2023-11-02T15:34:07.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 1 -- 172.21.15.174:0/221512757 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc02e070 con 0x7f10cc06b7d0 2023-11-02T15:34:07.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:07.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: dump: 2023-11-02T15:34:07.259 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.259 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.259 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.259 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.260 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 1 -- 172.21.15.174:0/221512757 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc004030 con 0x7f10cc06b7d0 2023-11-02T15:34:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 1 -- 172.21.15.174:0/221512757 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc004420 con 0x7f10cc06b7d0 2023-11-02T15:34:07.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:07.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10ca7fc640 10 monclient: dump: 2023-11-02T15:34:07.262 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.263 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.263 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.263 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.263 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.263 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.264 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.264 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.264 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.264 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 Processor -- start 2023-11-02T15:34:07.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10d1eb4640 5 monclient: authenticate success, global_id 4363 2023-11-02T15:34:07.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10d1eb4640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:07.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f10d1eb4640 4 set_mon_vals no callback set 2023-11-02T15:34:07.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 -- start start 2023-11-02T15:34:07.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 librados: setting wanted keys 2023-11-02T15:34:07.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 1 librados: calling monclient init 2023-11-02T15:34:07.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f54c4df0640 10 monclient: init 2023-11-02T15:34:07.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.189+0000 7f8c81e72640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7c13e520 0x7f8c7c1364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:07.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-b con 0x7f54c00b0ed0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:07.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 10 monclient: shutdown 2023-11-02T15:34:07.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 -- 172.21.15.174:0/221512757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 msgr2=0x7f10cc0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc0adab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009850 tx=0x7f10bc02b840 comp rx=0 tx=0).stop 2023-11-02T15:34:07.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54c00b1cb0 0x7f54c00b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 -- 172.21.15.174:0/221512757 shutdown_connections 2023-11-02T15:34:07.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0adff0 0x7f10cc144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/221512757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc0adab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 -- 172.21.15.174:0/221512757 >> 172.21.15.174:0/221512757 conn(0x7f10cc0b5060 msgr2=0x7f10cc0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81e72640 10 monclient(hunting): get_auth_request con 0x7f8c7c13e520 auth_method 0 2023-11-02T15:34:07.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81e72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81e72640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81e72640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-a con 0x7f54c00b1cb0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:07.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 -- 172.21.15.174:0/221512757 shutdown_connections 2023-11-02T15:34:07.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b4570 0x7f54c00b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f10d1eb4640 1 -- 172.21.15.174:0/221512757 wait complete. 2023-11-02T15:34:07.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): picked mon.noname-c con 0x7f54c00b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00ac7d0 con 0x7f54c00b1cb0 2023-11-02T15:34:07.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00ab8b0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c0142390 con 0x7f54c00b4570 2023-11-02T15:34:07.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f54c4df0640 10 monclient(hunting): authenticate will time out at 1374.532420s 2023-11-02T15:34:07.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 10 monclient(hunting): handle_auth_done global_id 4307 payload 824 2023-11-02T15:34:07.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 -- 172.21.15.174:0/3991103625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7c13e520 msgr2=0x7f8c7c1364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7c13e520 0x7f8c7c1364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 -- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4570 msgr2=0x7f8c7c1412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4570 0x7f8c7c1412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 monclient: found mon.noname-c 2023-11-02T15:34:07.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.190+0000 7f8c81671640 1 -- 172.21.15.174:0/3991103625 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74009b30 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.191+0000 7f10d1eb4640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.191+0000 7f10d1eb4640 10 monclient: monmap: 2023-11-02T15:34:07.282 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.282 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:07.282 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:07.282 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:07.283 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.283 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.283 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:07.283 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:07.284 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:07.284 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.191+0000 7f10d1eb4640 1 librados: starting msgr at 2023-11-02T15:34:07.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.191+0000 7f10d1eb4640 1 librados: starting objecter 2023-11-02T15:34:07.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.191+0000 7f10d1eb4640 1 Processor -- start 2023-11-02T15:34:07.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 -- start start 2023-11-02T15:34:07.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 librados: setting wanted keys 2023-11-02T15:34:07.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 librados: calling monclient init 2023-11-02T15:34:07.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 10 monclient: init 2023-11-02T15:34:07.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T15:34:07.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-b con 0x7f10cc06b7d0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:07.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0adff0 0x7f10cc1d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-a con 0x7f10cc0adff0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:07.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.192+0000 7f10d1eb4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0ae460 0x7f10cc1d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): picked mon.noname-c con 0x7f10cc0ae460 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc120d10 con 0x7f10cc0adff0 2023-11-02T15:34:07.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc1220e0 con 0x7f10cc06b7d0 2023-11-02T15:34:07.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc11fff0 con 0x7f10cc0ae460 2023-11-02T15:34:07.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10d1eb4640 10 monclient(hunting): authenticate will time out at 1374.534749s 2023-11-02T15:34:07.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f8c81671640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c440 tx=0x7f8c7800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f8c71ffb640 10 client.?.objecter ms_handle_connect 0x7f8c7c0b0ed0 2023-11-02T15:34:07.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f8c71ffb640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:07.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.193+0000 7f54be59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:48866/0 (socket says 172.21.15.174:48866) 2023-11-02T15:34:07.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:48850/0 (socket says 172.21.15.174:48850) 2023-11-02T15:34:07.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 1 -- 172.21.15.174:0/3649650975 learned_addr learned my addr 172.21.15.174:0/3649650975 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 10 monclient(hunting): get_auth_request con 0x7f54c00b0ed0 auth_method 0 2023-11-02T15:34:07.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f54be59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 1 -- 172.21.15.174:0/789607799 learned_addr learned my addr 172.21.15.174:0/789607799 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 10 monclient(hunting): get_auth_request con 0x7f10cc06b7d0 auth_method 0 2023-11-02T15:34:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.194+0000 7f10cb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f54be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f54be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.195+0000 7f54be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f54bed9d640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b4570 0x7f54c00b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f54bed9d640 10 monclient(hunting): get_auth_request con 0x7f54c00b4570 auth_method 0 2023-11-02T15:34:07.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f54bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f54bed9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f54bed9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.197+0000 7f10cbfff640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0ae460 0x7f10cc1d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 10 monclient(hunting): handle_auth_done global_id 4372 payload 824 2023-11-02T15:34:07.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 -- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0ae460 msgr2=0x7f10cc1d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0ae460 0x7f10cc1d6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 -- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0adff0 msgr2=0x7f10cc1d69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0adff0 0x7f10cc1d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 monclient: found mon.noname-b 2023-11-02T15:34:07.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f10cb7fe640 1 -- 172.21.15.174:0/789607799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc009460 con 0x7f10cc06b7d0 2023-11-02T15:34:07.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.198+0000 7f54bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78005480 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:07.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: dump: 2023-11-02T15:34:07.304 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.304 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.304 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.304 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.305 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.305 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.305 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.305 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.305 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.306 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c780035f0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780039c0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:07.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c71ffb640 10 monclient: dump: 2023-11-02T15:34:07.308 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.308 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.308 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.308 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.308 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.309 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.309 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.309 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.309 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.309 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c838d5640 5 monclient: authenticate success, global_id 4307 2023-11-02T15:34:07.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c838d5640 10 monclient: _renew_subs 2023-11-02T15:34:07.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c838d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c1420b0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.199+0000 7f8c737fe640 4 set_mon_vals no callback set 2023-11-02T15:34:07.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.200+0000 7f8c838d5640 10 client.4307.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:07.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.200+0000 7f8c838d5640 10 monclient: _renew_subs 2023-11-02T15:34:07.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.200+0000 7f8c838d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.200+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c136be0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.200+0000 7f8c838d5640 1 librados: init done 2023-11-02T15:34:07.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f8c838d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f8c838d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f8c838d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c067920 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 10 monclient(hunting): handle_auth_done global_id 4375 payload 824 2023-11-02T15:34:07.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 -- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b4570 msgr2=0x7f54c00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b4570 0x7f54c00b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 -- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54c00b1cb0 msgr2=0x7f54c00b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54c00b1cb0 0x7f54c00b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 monclient: found mon.noname-b 2023-11-02T15:34:07.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.201+0000 7f54be59c640 1 -- 172.21.15.174:0/3649650975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4009460 con 0x7f54c00b0ed0 2023-11-02T15:34:07.315 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.203+0000 7f10cb7fe640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10bc035040 tx=0x7f10bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.204+0000 7f10b3fff640 10 client.?.objecter ms_handle_connect 0x7f10cc06b7d0 2023-11-02T15:34:07.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.204+0000 7f10b3fff640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:07.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.207+0000 7f54be59c640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b440 tx=0x7f54b000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.207+0000 7f54aeffd640 10 client.?.objecter ms_handle_connect 0x7f54c00b0ed0 2023-11-02T15:34:07.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.207+0000 7f54aeffd640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:07.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc003c20 con 0x7f10cc06b7d0 2023-11-02T15:34:07.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:07.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: dump: 2023-11-02T15:34:07.320 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.320 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.320 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.320 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.320 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.321 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.321 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.321 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.321 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.321 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc003d80 con 0x7f10cc06b7d0 2023-11-02T15:34:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc006280 con 0x7f10cc06b7d0 2023-11-02T15:34:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10b3fff640 10 monclient: dump: 2023-11-02T15:34:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.324 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.324 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.324 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.324 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.324 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.325 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10c9ffb640 4 set_mon_vals no callback set 2023-11-02T15:34:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 5 monclient: authenticate success, global_id 4372 2023-11-02T15:34:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 10 monclient: _renew_subs 2023-11-02T15:34:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10cc140a40 con 0x7f10cc06b7d0 2023-11-02T15:34:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 10 client.4372.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 10 monclient: _renew_subs 2023-11-02T15:34:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc121dc0 con 0x7f10cc06b7d0 2023-11-02T15:34:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.209+0000 7f10d1eb4640 1 librados: init done 2023-11-02T15:34:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.210+0000 7f10d1eb4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.210+0000 7f10d1eb4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.210+0000 7f10d1eb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.210+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10cc140c30 con 0x7f10cc06b7d0 2023-11-02T15:34:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0011890 con 0x7f54c00b0ed0 2023-11-02T15:34:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: dump: 2023-11-02T15:34:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.332 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.332 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.332 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.332 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b00119f0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 5 monclient: authenticate success, global_id 4375 2023-11-02T15:34:07.333 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 10 monclient: _renew_subs 2023-11-02T15:34:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c00b0bb0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0011da0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:07.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54aeffd640 10 monclient: dump: 2023-11-02T15:34:07.335 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.335 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.335 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 10 client.4375.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 10 monclient: _renew_subs 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54bcd99640 4 set_mon_vals no callback set 2023-11-02T15:34:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c0139380 con 0x7f54c00b0ed0 2023-11-02T15:34:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.218+0000 7f54c4df0640 1 librados: init done 2023-11-02T15:34:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.219+0000 7f54c4df0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.219+0000 7f54c4df0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.219+0000 7f54c4df0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.219+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c0067920 con 0x7f54c00b0ed0 2023-11-02T15:34:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.295+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb77c0764f0 con 0x7fb78c0b3850 2023-11-02T15:34:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.295+0000 7fb788ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.295+0000 7fb788ff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:07.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.448+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f10bc039050 con 0x7f10cc06b7d0 2023-11-02T15:34:07.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.449+0000 7f10b3fff640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 0x7f10b806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.449+0000 7f10caffd640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 0x7f10b806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.449+0000 7f10caffd640 10 monclient: get_auth_request con 0x7f10b806be10 auth_method 0 2023-11-02T15:34:07.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.449+0000 7f10b3fff640 10 client.4372.objecter ms_handle_connect 0x7f10b806be10 2023-11-02T15:34:07.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.449+0000 7f10caffd640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 0x7f10b806e2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006fd0 tx=0x7f10c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.451+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f10bc045e40 con 0x7f10cc06b7d0 2023-11-02T15:34:07.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.451+0000 7f10b3fff640 10 client.4372.objecter ms_dispatch 0x7f10cc000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:07.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.451+0000 7f10b3fff640 3 client.4372.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:07.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.451+0000 7f10b3fff640 3 client.4372.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:07.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.451+0000 7f10b3fff640 20 client.4372.objecter dump_active .. 0 homeless 2023-11-02T15:34:07.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.469+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8c78003b20 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.469+0000 7f8c71ffb640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 0x7f8c6006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.470+0000 7f8c80e70640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 0x7f8c6006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.470+0000 7f8c80e70640 10 monclient: get_auth_request con 0x7f8c6006bdc0 auth_method 0 2023-11-02T15:34:07.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.470+0000 7f8c80e70640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 0x7f8c6006e280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006e20 tx=0x7f8c6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.470+0000 7f8c71ffb640 10 client.4307.objecter ms_handle_connect 0x7f8c6006bdc0 2023-11-02T15:34:07.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.471+0000 7fb79232e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:07.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.471+0000 7fb79232e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:07.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.471+0000 7fb79232e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T15:34:07.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.471+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb78c068b90 con 0x7fb78c0b3850 2023-11-02T15:34:07.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.476+0000 7fb788ff9640 1 -- 172.21.15.174:0/3430037646 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb77c035090 con 0x7fb78c0b3850 2023-11-02T15:34:07.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.476+0000 7fb788ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2023-11-02T15:34:07.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.476+0000 7fb788ff9640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:07.479 INFO:teuthology.orchestra.run.smithi174.stdout:60129542149 2023-11-02T15:34:07.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 10 librados: watch_flush enter 2023-11-02T15:34:07.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 10 librados: watch_flush exit 2023-11-02T15:34:07.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 20 client.4314.objecter shutdown clearing up homeless session... 2023-11-02T15:34:07.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 10 client.4314.objecter successfully canceled tick 2023-11-02T15:34:07.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 msgr2=0x7fb77806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 0x7fb77806e2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7800011e0 tx=0x7fb780007d30 comp rx=0 tx=0).stop 2023-11-02T15:34:07.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 10 monclient: shutdown 2023-11-02T15:34:07.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 msgr2=0x7fb78c13b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb77c034040 tx=0x7fb77c0046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 shutdown_connections 2023-11-02T15:34:07.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb77806be10 0x7fb77806e2d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb78c0b3c30 0x7fb78c13b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c0b3850 0x7fb78c13b3e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 --2- 172.21.15.174:0/3430037646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb78c0b0950 0x7fb78c1d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.478+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 >> 172.21.15.174:0/3430037646 conn(0x7fb78c065960 msgr2=0x7fb78c0b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.479+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 shutdown_connections 2023-11-02T15:34:07.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.479+0000 7fb79232e640 1 -- 172.21.15.174:0/3430037646 wait complete. 2023-11-02T15:34:07.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.479+0000 7fb79232e640 1 librados: shutdown 2023-11-02T15:34:07.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8c78057600 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 10 client.4307.objecter ms_dispatch 0x7f8c7c000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:07.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 3 client.4307.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:07.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 3 client.4307.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:07.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 20 client.4307.objecter dump_active .. 0 homeless 2023-11-02T15:34:07.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8c78056cd0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.485+0000 7f8c71ffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:07.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2023-11-02T15:34:07.493 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:07.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.660+0000 7f8c838d5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.660+0000 7f8c838d5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.660+0000 7f8c838d5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.660+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c7c0a8bb0 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.663+0000 7f8c71ffb640 1 -- 172.21.15.174:0/3991103625 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c7c0b0ed0 2023-11-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.663+0000 7f8c71ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2023-11-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.663+0000 7f8c71ffb640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:07.666 INFO:teuthology.orchestra.run.smithi174.stdout:68719476741 2023-11-02T15:34:07.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 10 librados: watch_flush enter 2023-11-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 10 librados: watch_flush exit 2023-11-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 20 client.4307.objecter shutdown clearing up homeless session... 2023-11-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 10 client.4307.objecter successfully canceled tick 2023-11-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 msgr2=0x7f8c6006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 0x7f8c6006e280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006e20 tx=0x7f8c6c008040 comp rx=0 tx=0).stop 2023-11-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 10 monclient: shutdown 2023-11-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 msgr2=0x7f8c7c0b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.665+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c440 tx=0x7f8c7800c900 comp rx=0 tx=0).stop 2023-11-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 shutdown_connections 2023-11-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7c13e520 0x7f8c7c1364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4570 0x7f8c7c1412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f8c6006bdc0 0x7f8c6006e280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 --2- 172.21.15.174:0/3991103625 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b0ed0 0x7f8c7c0b7220 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 >> 172.21.15.174:0/3991103625 conn(0x7f8c7c065d50 msgr2=0x7f8c7c0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 shutdown_connections 2023-11-02T15:34:07.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 -- 172.21.15.174:0/3991103625 wait complete. 2023-11-02T15:34:07.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.666+0000 7f8c838d5640 1 librados: shutdown 2023-11-02T15:34:07.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476741 for osd.2 2023-11-02T15:34:07.678 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:07.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.688+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f10bc075ee0 con 0x7f10cc06b7d0 2023-11-02T15:34:07.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.688+0000 7f10b3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.688+0000 7f10b3fff640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:07.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.703+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f54b001a400 con 0x7f54c00b0ed0 2023-11-02T15:34:07.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.704+0000 7f54aeffd640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 0x7f54a006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.704+0000 7f54bdd9b640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 0x7f54a006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.704+0000 7f54bdd9b640 10 monclient: get_auth_request con 0x7f54a006be10 auth_method 0 2023-11-02T15:34:07.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.704+0000 7f54aeffd640 10 client.4375.objecter ms_handle_connect 0x7f54a006be10 2023-11-02T15:34:07.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.704+0000 7f54bdd9b640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 0x7f54a006e2d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54b4035000 tx=0x7f54b4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.706+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f54b0058b60 con 0x7f54c00b0ed0 2023-11-02T15:34:07.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.706+0000 7f54aeffd640 10 client.4375.objecter ms_dispatch 0x7f54c0000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:07.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.706+0000 7f54aeffd640 3 client.4375.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:07.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.706+0000 7f54aeffd640 3 client.4375.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:07.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.706+0000 7f54aeffd640 20 client.4375.objecter dump_active .. 0 homeless 2023-11-02T15:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.716+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f54b00547a0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.716+0000 7f54aeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.716+0000 7f54aeffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:07.835 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-02T15:34:07.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.873+0000 7f10d1eb4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:07.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.873+0000 7f10d1eb4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:07.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.873+0000 7f10d1eb4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.873+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f10cc04e040 con 0x7f10cc06b7d0 2023-11-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.905+0000 7f10b3fff640 1 -- 172.21.15.174:0/789607799 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10bc037030 con 0x7f10cc06b7d0 2023-11-02T15:34:07.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.905+0000 7f10b3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2023-11-02T15:34:07.907 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.905+0000 7f10b3fff640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:07.907 INFO:teuthology.orchestra.run.smithi174.stdout:60129542149 2023-11-02T15:34:07.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.907+0000 7f54c4df0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:07.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.907+0000 7f54c4df0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:07.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.907+0000 7f54c4df0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:07.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.907+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54c00b21b0 con 0x7f54c00b0ed0 2023-11-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 10 librados: watch_flush enter 2023-11-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 10 librados: watch_flush exit 2023-11-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 20 client.4372.objecter shutdown clearing up homeless session... 2023-11-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 10 client.4372.objecter successfully canceled tick 2023-11-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 msgr2=0x7f10b806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 0x7f10b806e2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006fd0 tx=0x7f10c0008040 comp rx=0 tx=0).stop 2023-11-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 10 monclient: shutdown 2023-11-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 msgr2=0x7f10cc1216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10bc035040 tx=0x7f10bc0046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 shutdown_connections 2023-11-02T15:34:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f10b806be10 0x7f10b806e2d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc0ae460 0x7f10cc1d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0adff0 0x7f10cc1d69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 --2- 172.21.15.174:0/789607799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10cc06b7d0 0x7f10cc1216a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 >> 172.21.15.174:0/789607799 conn(0x7f10cc0b5060 msgr2=0x7f10cc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 shutdown_connections 2023-11-02T15:34:07.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 -- 172.21.15.174:0/789607799 wait complete. 2023-11-02T15:34:07.917 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.908+0000 7f10d1eb4640 1 librados: shutdown 2023-11-02T15:34:07.917 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.910+0000 7f54aeffd640 1 -- 172.21.15.174:0/3649650975 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54b0015090 con 0x7f54c00b0ed0 2023-11-02T15:34:07.918 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.910+0000 7f54aeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2023-11-02T15:34:07.918 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.910+0000 7f54aeffd640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:07.918 INFO:teuthology.orchestra.run.smithi174.stdout:68719476742 2023-11-02T15:34:07.919 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 10 librados: watch_flush enter 2023-11-02T15:34:07.919 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 10 librados: watch_flush exit 2023-11-02T15:34:07.919 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 20 client.4375.objecter shutdown clearing up homeless session... 2023-11-02T15:34:07.920 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 10 client.4375.objecter successfully canceled tick 2023-11-02T15:34:07.920 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 msgr2=0x7f54a006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.920 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 0x7f54a006e2d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54b4035000 tx=0x7f54b4030000 comp rx=0 tx=0).stop 2023-11-02T15:34:07.920 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 10 monclient: shutdown 2023-11-02T15:34:07.920 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.921 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 msgr2=0x7f54c00abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.921 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b440 tx=0x7f54b000b900 comp rx=0 tx=0).stop 2023-11-02T15:34:07.921 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 shutdown_connections 2023-11-02T15:34:07.921 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f54a006be10 0x7f54a006e2d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.921 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54c00b4570 0x7f54c00b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.922 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54c00b1cb0 0x7f54c00b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.922 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 --2- 172.21.15.174:0/3649650975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c00b0ed0 0x7f54c00abd20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 >> 172.21.15.174:0/3649650975 conn(0x7f54c0065d50 msgr2=0x7f54c013cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 shutdown_connections 2023-11-02T15:34:07.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 -- 172.21.15.174:0/3649650975 wait complete. 2023-11-02T15:34:07.923 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.912+0000 7f54c4df0640 1 librados: shutdown 2023-11-02T15:34:07.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476742 for osd.0 2023-11-02T15:34:07.928 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:07.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2023-11-02T15:34:07.929 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:07.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.946+0000 7f39ff83a640 10 monclient: get_monmap_and_config 2023-11-02T15:34:07.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.946+0000 7f39ff83a640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.946+0000 7f39ff83a640 10 monclient: monmap: 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:07.947637+0000 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:07.947637+0000 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.949 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.950 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:07.950 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:07.950 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:07.950 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:07.950 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:07.951 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.953 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.947+0000 7f39ff83a640 1 Processor -- start 2023-11-02T15:34:07.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.947+0000 7f39ff83a640 1 -- start start 2023-11-02T15:34:07.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.947+0000 7f39ff83a640 10 monclient: init 2023-11-02T15:34:07.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2023-11-02T15:34:07.954 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39f80b0ed0 0x7f39f8144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-c con 0x7f39f80b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:07.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-b con 0x7f39f80b4300 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f39f8070fb0 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:07.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80aa4b0 con 0x7f39f8070fb0 2023-11-02T15:34:07.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f80a8b50 con 0x7f39f80b0ed0 2023-11-02T15:34:07.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80b3a40 con 0x7f39f80b4300 2023-11-02T15:34:07.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39ff83a640 10 monclient(hunting): authenticate will time out at 1375.289886s 2023-11-02T15:34:07.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.948+0000 7f39fcdd5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39f80b0ed0 0x7f39f8144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:58088/0 (socket says 172.21.15.174:58088) 2023-11-02T15:34:07.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 1 -- 172.21.15.174:0/102382877 learned_addr learned my addr 172.21.15.174:0/102382877 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): get_auth_request con 0x7f39f80b4300 auth_method 0 2023-11-02T15:34:07.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): get_auth_request con 0x7f39f80b0ed0 auth_method 0 2023-11-02T15:34:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.949+0000 7f39fd5d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_done global_id 4313 payload 274 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 1 -- 172.21.15.174:0/102382877 >> v2:172.21.15.175:3300/0 conn(0x7f39f80b0ed0 msgr2=0x7f39f8144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 1 --2- 172.21.15.174:0/102382877 >> v2:172.21.15.175:3300/0 conn(0x7f39f80b0ed0 0x7f39f8144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 1 -- 172.21.15.174:0/102382877 >> v1:172.21.15.174:6789/0 conn(0x7f39f8070fb0 legacy=0x7f39f80b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:07.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 1 monclient: found mon.noname-b 2023-11-02T15:34:07.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.951+0000 7f39fcdd5640 1 -- 172.21.15.174:0/102382877 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80b3bb0 con 0x7f39f80b4300 2023-11-02T15:34:07.965 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.952+0000 7f39fcdd5640 1 --2- 172.21.15.174:0/102382877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009850 tx=0x7f39e802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d30085a5c28dfce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 1 -- 172.21.15.174:0/102382877 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e802e070 con 0x7f39f80b4300 2023-11-02T15:34:07.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:07.966 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: dump: 2023-11-02T15:34:07.967 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.967 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.968 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.969 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.969 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.969 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.969 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.969 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 1 -- 172.21.15.174:0/102382877 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8004030 con 0x7f39f80b4300 2023-11-02T15:34:07.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39ff83a640 5 monclient: authenticate success, global_id 4313 2023-11-02T15:34:07.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39ff83a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:07.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.970 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39ff83a640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:07.971 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39ff83a640 4 set_mon_vals no callback set 2023-11-02T15:34:07.971 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 1 -- 172.21.15.174:0/102382877 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8004420 con 0x7f39f80b4300 2023-11-02T15:34:07.971 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.971 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:07.971 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.956+0000 7f39effff640 10 monclient: dump: 2023-11-02T15:34:07.972 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.972 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.972 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.972 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.972 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.973 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 10 monclient: shutdown 2023-11-02T15:34:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 -- 172.21.15.174:0/102382877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 msgr2=0x7f39f80a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 --2- 172.21.15.174:0/102382877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009850 tx=0x7f39e802b840 comp rx=0 tx=0).stop 2023-11-02T15:34:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 -- 172.21.15.174:0/102382877 shutdown_connections 2023-11-02T15:34:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 --2- 172.21.15.174:0/102382877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f80a9070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 --2- 172.21.15.174:0/102382877 >> v2:172.21.15.175:3300/0 conn(0x7f39f80b0ed0 0x7f39f8144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 -- 172.21.15.174:0/102382877 >> 172.21.15.174:0/102382877 conn(0x7f39f8065d50 msgr2=0x7f39f8066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 -- 172.21.15.174:0/102382877 shutdown_connections 2023-11-02T15:34:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 1 -- 172.21.15.174:0/102382877 wait complete. 2023-11-02T15:34:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.957+0000 7f39ff83a640 10 monclient: build_initial_monmap 2023-11-02T15:34:07.976 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 10 monclient: monmap: 2023-11-02T15:34:07.976 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:07.976 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:07.976 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:07.976 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:07.977 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:07.977 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.978 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:07.978 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:07.978 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:07.978 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 librados: starting msgr at 2023-11-02T15:34:07.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 librados: starting objecter 2023-11-02T15:34:07.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 Processor -- start 2023-11-02T15:34:07.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 -- start start 2023-11-02T15:34:07.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 librados: setting wanted keys 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 1 librados: calling monclient init 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.958+0000 7f39ff83a640 10 monclient: init 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80b0ed0 0x7f39f80b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-a con 0x7f39f80b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:07.981 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.981 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-c con 0x7f39f80b4300 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:07.981 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f813f640 0x7f39f813fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:07.981 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): picked mon.noname-b con 0x7f39f813f640 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:07.981 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81427e0 con 0x7f39f80b0ed0 2023-11-02T15:34:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8140000 con 0x7f39f813f640 2023-11-02T15:34:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f806fcc0 con 0x7f39f80b4300 2023-11-02T15:34:07.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:07.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39ff83a640 10 monclient(hunting): authenticate will time out at 1375.300777s 2023-11-02T15:34:07.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:07.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:58092/0 (socket says 172.21.15.174:58092) 2023-11-02T15:34:07.983 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 1 -- 172.21.15.174:0/2114869493 learned_addr learned my addr 172.21.15.174:0/2114869493 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:07.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 10 monclient(hunting): get_auth_request con 0x7f39f80b4300 auth_method 0 2023-11-02T15:34:07.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:07.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:07.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.959+0000 7f39fcdd5640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:07.984 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.960+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.960+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.960+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 10 monclient(hunting): handle_auth_done global_id 4316 payload 824 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 -- 172.21.15.174:0/2114869493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f813f640 msgr2=0x7f39f813fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f813f640 0x7f39f813fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.985 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 -- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80b0ed0 msgr2=0x7f39f80b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:07.986 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80b0ed0 0x7f39f80b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:07.986 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 10 monclient: _finish_hunting 0 2023-11-02T15:34:07.986 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 monclient: found mon.noname-c 2023-11-02T15:34:07.986 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:07.986 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:07.987 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.963+0000 7f39fcdd5640 1 -- 172.21.15.174:0/2114869493 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8009460 con 0x7f39f80b4300 2023-11-02T15:34:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.965+0000 7f39fcdd5640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39e8035040 tx=0x7f39e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.965+0000 7f39edffb640 10 client.?.objecter ms_handle_connect 0x7f39f80b4300 2023-11-02T15:34:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.965+0000 7f39edffb640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8003aa0 con 0x7f39f80b4300 2023-11-02T15:34:07.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:07.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: dump: 2023-11-02T15:34:07.990 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.990 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.991 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.991 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:07.991 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:07.991 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:07.991 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: _finish_auth 0 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: _check_auth_tickets 2023-11-02T15:34:07.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:07.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8003c00 con 0x7f39f80b4300 2023-11-02T15:34:07.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:07.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8006040 con 0x7f39f80b4300 2023-11-02T15:34:07.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:07.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:07.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39edffb640 10 monclient: dump: 2023-11-02T15:34:07.994 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:07.995 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:07.995 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.002 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.002 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:08.003 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.003 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:08.003 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:08.003 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:08.003 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39ff83a640 5 monclient: authenticate success, global_id 4316 2023-11-02T15:34:08.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39ff83a640 10 monclient: _renew_subs 2023-11-02T15:34:08.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39ef7fe640 4 set_mon_vals no callback set 2023-11-02T15:34:08.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39ff83a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.968+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8142490 con 0x7f39f80b4300 2023-11-02T15:34:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.969+0000 7f39ff83a640 10 client.4316.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.969+0000 7f39ff83a640 10 monclient: _renew_subs 2023-11-02T15:34:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.969+0000 7f39ff83a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.969+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81394f0 con 0x7f39f80b4300 2023-11-02T15:34:08.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.969+0000 7f39ff83a640 1 librados: init done 2023-11-02T15:34:08.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.970+0000 7f39ff83a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.970+0000 7f39ff83a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.970+0000 7f39ff83a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.014 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.970+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8141a00 con 0x7f39f80b4300 2023-11-02T15:34:08.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.989+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f39e80061a0 con 0x7f39f80b4300 2023-11-02T15:34:08.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.990+0000 7f39edffb640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 0x7f39dc06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.990+0000 7f39fd5d6640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 0x7f39dc06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.990+0000 7f39fd5d6640 10 monclient: get_auth_request con 0x7f39dc06bdc0 auth_method 0 2023-11-02T15:34:08.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.990+0000 7f39edffb640 10 client.4316.objecter ms_handle_connect 0x7f39dc06bdc0 2023-11-02T15:34:08.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.990+0000 7f39fd5d6640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 0x7f39dc06e280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39f4006a90 tx=0x7f39f4006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:08.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.993+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f39e807a8d0 con 0x7f39f80b4300 2023-11-02T15:34:08.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.993+0000 7f39edffb640 10 client.4316.objecter ms_dispatch 0x7f39f8000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:08.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.993+0000 7f39edffb640 3 client.4316.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:08.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.993+0000 7f39edffb640 3 client.4316.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:08.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:07.993+0000 7f39edffb640 20 client.4316.objecter dump_active .. 0 homeless 2023-11-02T15:34:08.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.227+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f39e8076510 con 0x7f39f80b4300 2023-11-02T15:34:08.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.227+0000 7f39edffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.227+0000 7f39edffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:08.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.400+0000 7f39ff83a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:08.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.400+0000 7f39ff83a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:08.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.400+0000 7f39ff83a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.400+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f39f8141a00 con 0x7f39f80b4300 2023-11-02T15:34:08.405 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.404+0000 7f39edffb640 1 -- 172.21.15.174:0/2114869493 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39f8141a00 con 0x7f39f80b4300 2023-11-02T15:34:08.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.404+0000 7f39edffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2023-11-02T15:34:08.406 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.404+0000 7f39edffb640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:08.406 INFO:teuthology.orchestra.run.smithi174.stdout:55834574854 2023-11-02T15:34:08.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 10 librados: watch_flush enter 2023-11-02T15:34:08.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 10 librados: watch_flush exit 2023-11-02T15:34:08.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 20 client.4316.objecter shutdown clearing up homeless session... 2023-11-02T15:34:08.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 10 client.4316.objecter successfully canceled tick 2023-11-02T15:34:08.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 msgr2=0x7f39dc06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 0x7f39dc06e280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39f4006a90 tx=0x7f39f4006a20 comp rx=0 tx=0).stop 2023-11-02T15:34:08.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 10 monclient: shutdown 2023-11-02T15:34:08.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:08.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 msgr2=0x7f39f8141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39e8035040 tx=0x7f39e80046c0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 shutdown_connections 2023-11-02T15:34:08.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39f813f640 0x7f39f813fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f39dc06bdc0 0x7f39dc06e280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80b4300 0x7f39f8141f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 --2- 172.21.15.174:0/2114869493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80b0ed0 0x7f39f80b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 >> 172.21.15.174:0/2114869493 conn(0x7f39f8065d50 msgr2=0x7f39f8066130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:08.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 shutdown_connections 2023-11-02T15:34:08.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 -- 172.21.15.174:0/2114869493 wait complete. 2023-11-02T15:34:08.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.406+0000 7f39ff83a640 1 librados: shutdown 2023-11-02T15:34:08.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-11-02T15:34:08.425 DEBUG:teuthology.parallel:result is None 2023-11-02T15:34:08.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-02T15:34:08.426 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T15:34:08.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.563+0000 7fa6e2f9f640 10 monclient: get_monmap_and_config 2023-11-02T15:34:08.565 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.563+0000 7fa6e2f9f640 10 monclient: build_initial_monmap 2023-11-02T15:34:08.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.563+0000 7fa6e2f9f640 10 monclient: monmap: 2023-11-02T15:34:08.566 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:08.566 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:08.566 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:08.564574+0000 2023-11-02T15:34:08.567 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:08.564574+0000 2023-11-02T15:34:08.567 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:08.567 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.568 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:08.568 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:08.568 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:08.568 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:08.569 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:08.569 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.571 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.564+0000 7fa6e2f9f640 1 Processor -- start 2023-11-02T15:34:08.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.564+0000 7fa6e2f9f640 1 -- start start 2023-11-02T15:34:08.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.564+0000 7fa6e2f9f640 10 monclient: init 2023-11-02T15:34:08.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.564+0000 7fa6e2f9f640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:08.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2023-11-02T15:34:08.572 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa6dc0aa4a0 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:08.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6dc0abe80 0x7fa6dc0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-c con 0x7fa6dc0abe80 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:08.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa6dc0af150 0x7fa6dc13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-a con 0x7fa6dc0af150 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:08.573 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc06aea0 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.574 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:08.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc0aa880 con 0x7fa6dc0af150 2023-11-02T15:34:08.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:08.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc071250 con 0x7fa6dc0abe80 2023-11-02T15:34:08.575 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:08.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e2f9f640 10 monclient(hunting): authenticate will time out at 1375.906746s 2023-11-02T15:34:08.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa6dc0af150 0x7fa6dc13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6dbfff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6dc0abe80 0x7fa6dc0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa6dc0af150 0x7fa6dc13cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39520/0 (socket says 172.21.15.174:39520) 2023-11-02T15:34:08.576 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 1 -- 172.21.15.174:0/4113058803 learned_addr learned my addr 172.21.15.174:0/4113058803 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:08.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 10 monclient(hunting): get_auth_request con 0x7fa6dc0af150 auth_method 0 2023-11-02T15:34:08.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:08.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:08.577 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.565+0000 7fa6e153c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:08.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6dbfff640 10 monclient(hunting): get_auth_request con 0x7fa6dc0abe80 auth_method 0 2023-11-02T15:34:08.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:08.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6dbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:08.578 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6dbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:08.579 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:08.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:08.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.566+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:08.580 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.567+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:08.581 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.567+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:08.581 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.567+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:08.581 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 10 monclient(hunting): handle_auth_done global_id 4317 payload 274 2023-11-02T15:34:08.582 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 1 -- 172.21.15.174:0/4113058803 >> v2:172.21.15.175:3300/0 conn(0x7fa6dc0abe80 msgr2=0x7fa6dc0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:08.582 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 1 --2- 172.21.15.174:0/4113058803 >> v2:172.21.15.175:3300/0 conn(0x7fa6dc0abe80 0x7fa6dc0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 1 -- 172.21.15.174:0/4113058803 >> v1:172.21.15.175:6789/0 conn(0x7fa6dc0aa4a0 legacy=0x7fa6dc06a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 10 monclient: _finish_hunting 0 2023-11-02T15:34:08.583 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 1 monclient: found mon.noname-a 2023-11-02T15:34:08.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:08.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T15:34:08.584 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.568+0000 7fa6e153c640 1 -- 172.21.15.174:0/4113058803 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0713c0 con 0x7fa6dc0af150 2023-11-02T15:34:08.585 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.570+0000 7fa6e153c640 1 --2- 172.21.15.174:0/4113058803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0af150 0x7fa6dc13cf50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0ab4a0 tx=0x7fa6d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27598b6e326e035 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:08.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 1 -- 172.21.15.174:0/4113058803 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d402ca80 con 0x7fa6dc0af150 2023-11-02T15:34:08.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:08.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T15:34:08.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: dump: 2023-11-02T15:34:08.597 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:08.597 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:08.598 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.598 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.599 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:08.599 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.599 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:08.600 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:08.600 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:08.600 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: _finish_auth 0 2023-11-02T15:34:08.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: _check_auth_tickets 2023-11-02T15:34:08.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:08.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 1 -- 172.21.15.174:0/4113058803 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d402cbe0 con 0x7fa6dc0af150 2023-11-02T15:34:08.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6e2f9f640 5 monclient: authenticate success, global_id 4317 2023-11-02T15:34:08.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6e2f9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:08.609 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:08.609 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6e2f9f640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:08.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 1 -- 172.21.15.174:0/4113058803 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40343f0 con 0x7fa6dc0af150 2023-11-02T15:34:08.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6e2f9f640 4 set_mon_vals no callback set 2023-11-02T15:34:08.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:08.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T15:34:08.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.573+0000 7fa6db7fe640 10 monclient: dump: 2023-11-02T15:34:08.611 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:08.611 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:08.612 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.612 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.612 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:08.612 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.612 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:08.613 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:08.613 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:08.613 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 10 monclient: shutdown 2023-11-02T15:34:08.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:08.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/4113058803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0af150 msgr2=0x7fa6dc13cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/4113058803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0af150 0x7fa6dc13cf50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0ab4a0 tx=0x7fa6d402bee0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/4113058803 shutdown_connections 2023-11-02T15:34:08.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/4113058803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0af150 0x7fa6dc13cf50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.614 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/4113058803 >> v2:172.21.15.175:3300/0 conn(0x7fa6dc0abe80 0x7fa6dc0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/4113058803 >> 172.21.15.174:0/4113058803 conn(0x7fa6dc069c70 msgr2=0x7fa6dc06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:08.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/4113058803 shutdown_connections 2023-11-02T15:34:08.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.574+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/4113058803 wait complete. 2023-11-02T15:34:08.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 10 monclient: build_initial_monmap 2023-11-02T15:34:08.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 10 monclient: monmap: 2023-11-02T15:34:08.616 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:08.616 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:08.616 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:08.616 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:08.616 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:08.617 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.617 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:08.617 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:08.617 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:08.618 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 librados: starting msgr at 2023-11-02T15:34:08.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 librados: starting objecter 2023-11-02T15:34:08.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 Processor -- start 2023-11-02T15:34:08.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 -- start start 2023-11-02T15:34:08.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 librados: setting wanted keys 2023-11-02T15:34:08.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 1 librados: calling monclient init 2023-11-02T15:34:08.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.575+0000 7fa6e2f9f640 10 monclient: init 2023-11-02T15:34:08.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:08.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:34:08.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 0x7fa6dc137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-c con 0x7fa6dc0aa4a0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:08.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 0x7fa6dc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-a con 0x7fa6dc0abe80 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:08.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6dc0af150 0x7fa6dc070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): picked mon.noname-b con 0x7fa6dc0af150 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:08.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:08.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc06f570 con 0x7fa6dc0abe80 2023-11-02T15:34:08.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc06e920 con 0x7fa6dc0af150 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc06eb90 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e2f9f640 10 monclient(hunting): authenticate will time out at 1375.917927s 2023-11-02T15:34:08.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 0x7fa6dc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6e0d3b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 0x7fa6dc137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 0x7fa6dc06fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39532/0 (socket says 172.21.15.174:39532) 2023-11-02T15:34:08.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 1 -- 172.21.15.174:0/2154119633 learned_addr learned my addr 172.21.15.174:0/2154119633 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:08.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 10 monclient(hunting): get_auth_request con 0x7fa6dc0abe80 auth_method 0 2023-11-02T15:34:08.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:08.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:08.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.576+0000 7fa6dbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:08.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): get_auth_request con 0x7fa6dc0aa4a0 auth_method 0 2023-11-02T15:34:08.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:08.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:08.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:08.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6dc0af150 0x7fa6dc070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:08.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:08.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:08.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): get_auth_request con 0x7fa6dc0af150 auth_method 0 2023-11-02T15:34:08.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:08.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:08.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:08.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:08.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:08.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e0d3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:08.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:08.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:08.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.577+0000 7fa6e153c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:08.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 10 monclient(hunting): handle_auth_done global_id 4319 payload 824 2023-11-02T15:34:08.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 -- 172.21.15.174:0/2154119633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6dc0af150 msgr2=0x7fa6dc070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6dc0af150 0x7fa6dc070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 -- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 msgr2=0x7fa6dc06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:08.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 0x7fa6dc06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:08.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 10 monclient: _finish_hunting 0 2023-11-02T15:34:08.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 monclient: found mon.noname-c 2023-11-02T15:34:08.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:08.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.580+0000 7fa6e0d3b640 1 -- 172.21.15.174:0/2154119633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d4009b30 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.582+0000 7fa6e0d3b640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 0x7fa6dc137da0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c3c0 tx=0x7fa6d000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:08.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.582+0000 7fa6d97fa640 10 client.?.objecter ms_handle_connect 0x7fa6dc0aa4a0 2023-11-02T15:34:08.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.582+0000 7fa6d97fa640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:08.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00052d0 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:08.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:08.636 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: dump: 2023-11-02T15:34:08.636 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:08.636 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:08.636 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.636 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.637 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:08.637 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.637 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:08.637 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:08.637 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:08.638 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: _finish_auth 0 2023-11-02T15:34:08.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: _check_auth_tickets 2023-11-02T15:34:08.638 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:08.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d0005430 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.640 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 5 monclient: authenticate success, global_id 4319 2023-11-02T15:34:08.640 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 10 monclient: _renew_subs 2023-11-02T15:34:08.640 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc06ee70 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:08.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0003830 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:08.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6d97fa640 10 monclient: dump: 2023-11-02T15:34:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:08.645 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:08.645 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:08.645 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:08.645 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:08.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6daffd640 4 set_mon_vals no callback set 2023-11-02T15:34:08.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 10 client.4319.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:08.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 10 monclient: _renew_subs 2023-11-02T15:34:08.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc06f200 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.586+0000 7fa6e2f9f640 1 librados: init done 2023-11-02T15:34:08.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.587+0000 7fa6e2f9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.587+0000 7fa6e2f9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.587+0000 7fa6e2f9f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:08.648 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.587+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc0b1dc0 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.605+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa6d0003990 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.606+0000 7fa6d97fa640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 0x7fa6c006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:08.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.606+0000 7fa6dbfff640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 0x7fa6c006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:08.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.606+0000 7fa6dbfff640 10 monclient: get_auth_request con 0x7fa6c006bdc0 auth_method 0 2023-11-02T15:34:08.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.606+0000 7fa6dbfff640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 0x7fa6c006e280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc070a60 tx=0x7fa6c800a2a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:08.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.606+0000 7fa6d97fa640 10 client.4319.objecter ms_handle_connect 0x7fa6c006bdc0 2023-11-02T15:34:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa6d0057320 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 10 client.4319.objecter ms_dispatch 0x7fa6dc000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 3 client.4319.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 3 client.4319.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 20 client.4319.objecter dump_active .. 0 homeless 2023-11-02T15:34:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa6d0053660 con 0x7fa6dc0aa4a0 2023-11-02T15:34:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:08.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:08.844+0000 7fa6d97fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:09.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.051+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6dc04e040 con 0x7fa6c006bdc0 2023-11-02T15:34:09.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.052+0000 7fa6d97fa640 1 -- 172.21.15.174:0/2154119633 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48061 (secure 0 0 0) 0x7fa6dc04e040 con 0x7fa6c006bdc0 2023-11-02T15:34:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:09.055 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T15:34:09.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 10 librados: watch_flush enter 2023-11-02T15:34:09.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 10 librados: watch_flush exit 2023-11-02T15:34:09.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 20 client.4319.objecter shutdown clearing up homeless session... 2023-11-02T15:34:09.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 10 client.4319.objecter successfully canceled tick 2023-11-02T15:34:09.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 msgr2=0x7fa6c006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 0x7fa6c006e280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc070a60 tx=0x7fa6c800a2a0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 10 monclient: shutdown 2023-11-02T15:34:09.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:09.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.055+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 msgr2=0x7fa6dc137da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 0x7fa6dc137da0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c3c0 tx=0x7fa6d000c880 comp rx=0 tx=0).stop 2023-11-02T15:34:09.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 shutdown_connections 2023-11-02T15:34:09.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6dc0af150 0x7fa6dc070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6dc0abe80 0x7fa6dc06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa6c006bdc0 0x7fa6c006e280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 --2- 172.21.15.174:0/2154119633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6dc0aa4a0 0x7fa6dc137da0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 >> 172.21.15.174:0/2154119633 conn(0x7fa6dc069c70 msgr2=0x7fa6dc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:09.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 shutdown_connections 2023-11-02T15:34:09.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 -- 172.21.15.174:0/2154119633 wait complete. 2023-11-02T15:34:09.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.056+0000 7fa6e2f9f640 1 librados: shutdown 2023-11-02T15:34:09.072 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-02T15:34:08.709416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218284,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756436,"statfs":{"total":773094113280,"available":772870590464,"internally_reserved":0,"allocated":2539520,"data_stored":1393240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.962258+0000","last_change":"2023-11-02T15:34:01.938543+0000","last_active":"2023-11-02T15:34:01.962258+0000","last_peered":"2023-11-02T15:34:01.962258+0000","last_clean":"2023-11-02T15:34:01.962258+0000","last_became_active":"2023-11-02T15:34:01.933504+0000","last_became_peered":"2023-11-02T15:34:01.933504+0000","last_unstale":"2023-11-02T15:34:01.962258+0000","last_undegraded":"2023-11-02T15:34:01.962258+0000","last_fullsized":"2023-11-02T15:34:01.962258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:09:55.163100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.050836+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T15:34:02.050836+0000","last_peered":"2023-11-02T15:34:02.050836+0000","last_clean":"2023-11-02T15:34:02.050836+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T15:34:02.050836+0000","last_undegraded":"2023-11-02T15:34:02.050836+0000","last_fullsized":"2023-11-02T15:34:02.050836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.787486+0000","last_change":"2023-11-02T15:34:01.592738+0000","last_active":"2023-11-02T15:34:01.787486+0000","last_peered":"2023-11-02T15:34:01.787486+0000","last_clean":"2023-11-02T15:34:01.787486+0000","last_became_active":"2023-11-02T15:34:01.583822+0000","last_became_peered":"2023-11-02T15:34:01.583822+0000","last_unstale":"2023-11-02T15:34:01.787486+0000","last_undegraded":"2023-11-02T15:34:01.787486+0000","last_fullsized":"2023-11-02T15:34:01.787486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:23:43.663036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.024969+0000","last_change":"2023-11-02T15:34:01.994993+0000","last_active":"2023-11-02T15:34:02.024969+0000","last_peered":"2023-11-02T15:34:02.024969+0000","last_clean":"2023-11-02T15:34:02.024969+0000","last_became_active":"2023-11-02T15:34:01.966537+0000","last_became_peered":"2023-11-02T15:34:01.966537+0000","last_unstale":"2023-11-02T15:34:02.024969+0000","last_undegraded":"2023-11-02T15:34:02.024969+0000","last_fullsized":"2023-11-02T15:34:02.024969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:54:18.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.964006+0000","last_change":"2023-11-02T15:34:01.955839+0000","last_active":"2023-11-02T15:34:01.964006+0000","last_peered":"2023-11-02T15:34:01.964006+0000","last_clean":"2023-11-02T15:34:01.964006+0000","last_became_active":"2023-11-02T15:34:01.950053+0000","last_became_peered":"2023-11-02T15:34:01.950053+0000","last_unstale":"2023-11-02T15:34:01.964006+0000","last_undegraded":"2023-11-02T15:34:01.964006+0000","last_fullsized":"2023-11-02T15:34:01.964006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:08:54.230636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.533782+0000","last_change":"2023-11-02T15:34:01.430741+0000","last_active":"2023-11-02T15:34:02.533782+0000","last_peered":"2023-11-02T15:34:02.533782+0000","last_clean":"2023-11-02T15:34:02.533782+0000","last_became_active":"2023-11-02T15:34:01.403797+0000","last_became_peered":"2023-11-02T15:34:01.403797+0000","last_unstale":"2023-11-02T15:34:02.533782+0000","last_undegraded":"2023-11-02T15:34:02.533782+0000","last_fullsized":"2023-11-02T15:34:02.533782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:20:09.696671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.775554+0000","last_change":"2023-11-02T15:34:01.461505+0000","last_active":"2023-11-02T15:34:01.775554+0000","last_peered":"2023-11-02T15:34:01.775554+0000","last_clean":"2023-11-02T15:34:01.775554+0000","last_became_active":"2023-11-02T15:34:01.430918+0000","last_became_peered":"2023-11-02T15:34:01.430918+0000","last_unstale":"2023-11-02T15:34:01.775554+0000","last_undegraded":"2023-11-02T15:34:01.775554+0000","last_fullsized":"2023-11-02T15:34:01.775554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:33:03.165716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.831688+0000","last_change":"2023-11-02T15:34:01.826046+0000","last_active":"2023-11-02T15:34:01.831688+0000","last_peered":"2023-11-02T15:34:01.831688+0000","last_clean":"2023-11-02T15:34:01.831688+0000","last_became_active":"2023-11-02T15:34:01.817659+0000","last_became_peered":"2023-11-02T15:34:01.817659+0000","last_unstale":"2023-11-02T15:34:01.831688+0000","last_undegraded":"2023-11-02T15:34:01.831688+0000","last_fullsized":"2023-11-02T15:34:01.831688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:16:14.944485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"22'76","reported_seq":117,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-02T15:34:06.971301+0000","last_change":"2023-11-02T15:33:55.693910+0000","last_active":"2023-11-02T15:34:06.971301+0000","last_peered":"2023-11-02T15:34:06.971301+0000","last_clean":"2023-11-02T15:34:06.971301+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T15:34:06.971301+0000","last_undegraded":"2023-11-02T15:34:06.971301+0000","last_fullsized":"2023-11-02T15:34:06.971301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_clean_scrub_stamp":"2023-11-02T15:33:53.161912+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:29.329385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.074999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.853999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.097000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.236000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.98}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":28.859999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":618496,"data_stored":472535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.255000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8599999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.787000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.18}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.353999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.429}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0220000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.433999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.16}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.731999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.254000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.252000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5220000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0370000000000008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.723000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":618496,"data_stored":472535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.3029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.806000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.617000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.528}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.483000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.215999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.438000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1559999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.023}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.021000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.57}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":26.609000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.7640000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.838000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.510999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.002000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.446000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.530999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6509999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.625999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.602}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.577}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.864999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T15:34:09.074 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T15:34:09.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 10 monclient: get_monmap_and_config 2023-11-02T15:34:09.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 10 monclient: build_initial_monmap 2023-11-02T15:34:09.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 10 monclient: monmap: 2023-11-02T15:34:09.182 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:09.182 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.183 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:09.174058+0000 2023-11-02T15:34:09.184 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:09.174058+0000 2023-11-02T15:34:09.184 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:09.184 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.184 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:09.185 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:09.185 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:09.185 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:09.185 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:09.186 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 1 Processor -- start 2023-11-02T15:34:09.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 1 -- start start 2023-11-02T15:34:09.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.173+0000 7f3fbafa2640 10 monclient: init 2023-11-02T15:34:09.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:09.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2023-11-02T15:34:09.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb40b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-b con 0x7f3fb40b3cd0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:09.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3fb40b40b0 addr v1:172.21.15.175:6789/0 2023-11-02T15:34:09.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 0x7f3fb40b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-a con 0x7f3fb40a84b0 addr v2:172.21.15.174:3300/0 2023-11-02T15:34:09.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb40b1500 con 0x7f3fb40b40b0 2023-11-02T15:34:09.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb40a8900 con 0x7f3fb40a84b0 2023-11-02T15:34:09.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40a8a40 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:09.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.174+0000 7f3fbafa2640 10 monclient(hunting): authenticate will time out at 1376.516184s 2023-11-02T15:34:09.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fba7a1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 0x7f3fb40b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb9fa0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb40b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb979f640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3fb40b40b0 0x7f3fb4144a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:41698/0 (socket says 172.21.15.174:41698) 2023-11-02T15:34:09.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb979f640 1 -- 172.21.15.174:0/1230922147 learned_addr learned my addr 172.21.15.174:0/1230922147 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:09.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fba7a1640 10 monclient(hunting): get_auth_request con 0x7f3fb40a84b0 auth_method 0 2023-11-02T15:34:09.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fba7a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fba7a1640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fba7a1640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb9fa0640 10 monclient(hunting): get_auth_request con 0x7f3fb40b3cd0 auth_method 0 2023-11-02T15:34:09.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb9fa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb9fa0640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.175+0000 7f3fb9fa0640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.176+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 1 -- 172.21.15.174:0/1230922147 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739885169 0 0) 0x7f3fb40b1500 con 0x7f3fb40b40b0 2023-11-02T15:34:09.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 10 monclient(hunting): _negotiate 2023-11-02T15:34:09.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 10 monclient(hunting): my global_id is 4390 2023-11-02T15:34:09.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.177+0000 7f3fb8f9e640 1 -- 172.21.15.174:0/1230922147 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa0003520 con 0x7f3fb40b40b0 2023-11-02T15:34:09.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_done global_id 4322 payload 274 2023-11-02T15:34:09.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/1230922147 >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 msgr2=0x7f3fb40b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:09.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 1 --2- 172.21.15.174:0/1230922147 >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 0x7f3fb40b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/1230922147 >> v1:172.21.15.175:6789/0 conn(0x7f3fb40b40b0 legacy=0x7f3fb4144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 10 monclient: _finish_hunting 0 2023-11-02T15:34:09.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 1 monclient: found mon.noname-b 2023-11-02T15:34:09.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:09.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/1230922147 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb40b0640 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.178+0000 7f3fba7a1640 1 --2- 172.21.15.174:0/1230922147 >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 0x7f3fb40b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T15:34:09.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.180+0000 7f3fb9fa0640 1 --2- 172.21.15.174:0/1230922147 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb40b0fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009d90 tx=0x7f3fac02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c73a6da67ff98d server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 1 -- 172.21.15.174:0/1230922147 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac02cc50 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:09.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: dump: 2023-11-02T15:34:09.208 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.208 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.208 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.208 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.209 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.209 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.209 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.209 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.209 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.210 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: _finish_auth 0 2023-11-02T15:34:09.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: _check_auth_tickets 2023-11-02T15:34:09.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:09.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 1 -- 172.21.15.174:0/1230922147 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac02cdb0 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fbafa2640 5 monclient: authenticate success, global_id 4322 2023-11-02T15:34:09.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fbafa2640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:09.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:09.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fbafa2640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:09.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 1 -- 172.21.15.174:0/1230922147 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac034660 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fbafa2640 4 set_mon_vals no callback set 2023-11-02T15:34:09.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:09.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.184+0000 7f3fb8f9e640 10 monclient: dump: 2023-11-02T15:34:09.213 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.213 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.213 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.213 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.214 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.215 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.215 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.215 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.215 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.215 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 10 monclient: shutdown 2023-11-02T15:34:09.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:09.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 -- 172.21.15.174:0/1230922147 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 msgr2=0x7f3fb40b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/1230922147 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb40b0fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009d90 tx=0x7f3fac02bea0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 -- 172.21.15.174:0/1230922147 shutdown_connections 2023-11-02T15:34:09.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/1230922147 >> v2:172.21.15.174:3300/0 conn(0x7f3fb40a84b0 0x7f3fb40b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/1230922147 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb40b0fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 -- 172.21.15.174:0/1230922147 >> 172.21.15.174:0/1230922147 conn(0x7f3fb4065b40 msgr2=0x7f3fb4065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:09.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 -- 172.21.15.174:0/1230922147 shutdown_connections 2023-11-02T15:34:09.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 1 -- 172.21.15.174:0/1230922147 wait complete. 2023-11-02T15:34:09.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 10 monclient: build_initial_monmap 2023-11-02T15:34:09.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.185+0000 7f3fbafa2640 10 monclient: monmap: 2023-11-02T15:34:09.218 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:09.219 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:09.219 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:09.219 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:09.219 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:09.219 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.220 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:09.220 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:09.220 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:09.220 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 librados: starting msgr at 2023-11-02T15:34:09.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 librados: starting objecter 2023-11-02T15:34:09.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 Processor -- start 2023-11-02T15:34:09.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 -- start start 2023-11-02T15:34:09.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 librados: setting wanted keys 2023-11-02T15:34:09.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 librados: calling monclient init 2023-11-02T15:34:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient: init 2023-11-02T15:34:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T15:34:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 0x7f3fb413d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-b con 0x7f3fb40a84b0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-c con 0x7f3fb40b3cd0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 0x7f3fb4139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): picked mon.noname-a con 0x7f3fb4139280 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb413e130 con 0x7f3fb4139280 2023-11-02T15:34:09.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb413e550 con 0x7f3fb40a84b0 2023-11-02T15:34:09.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb413c6e0 con 0x7f3fb40b3cd0 2023-11-02T15:34:09.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:09.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.186+0000 7f3fbafa2640 10 monclient(hunting): authenticate will time out at 1376.528540s 2023-11-02T15:34:09.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 0x7f3fb4139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb9fa0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 0x7f3fb413d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb979f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 0x7f3fb4139710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39550/0 (socket says 172.21.15.174:39550) 2023-11-02T15:34:09.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 1 -- 172.21.15.174:0/2034322264 learned_addr learned my addr 172.21.15.174:0/2034322264 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:09.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 10 monclient(hunting): get_auth_request con 0x7f3fb4139280 auth_method 0 2023-11-02T15:34:09.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fba7a1640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb979f640 10 monclient(hunting): get_auth_request con 0x7f3fb40b3cd0 auth_method 0 2023-11-02T15:34:09.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb979f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb979f640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb979f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb9fa0640 10 monclient(hunting): get_auth_request con 0x7f3fb40a84b0 auth_method 0 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb9fa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb9fa0640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.187+0000 7f3fb9fa0640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fba7a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb979f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb979f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.188+0000 7f3fb979f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 10 monclient(hunting): handle_auth_done global_id 4393 payload 824 2023-11-02T15:34:09.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 msgr2=0x7f3fb413d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb413d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 msgr2=0x7f3fb4139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 0x7f3fb4139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 10 monclient: _finish_hunting 0 2023-11-02T15:34:09.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 monclient: found mon.noname-b 2023-11-02T15:34:09.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:09.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:34:09.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.190+0000 7f3fb9fa0640 1 -- 172.21.15.174:0/2034322264 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0009400 con 0x7f3fb40a84b0 2023-11-02T15:34:09.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.193+0000 7f3fb9fa0640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 0x7f3fb413d2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fac033000 tx=0x7f3fac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.193+0000 7f3faa7fc640 10 client.?.objecter ms_handle_connect 0x7f3fb40a84b0 2023-11-02T15:34:09.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.193+0000 7f3faa7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:09.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac03e610 con 0x7f3fb40a84b0 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: dump: 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.233 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.234 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.234 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.234 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.234 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.234 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.235 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.235 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:09.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:09.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:09.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac03e7b0 con 0x7f3fb40a84b0 2023-11-02T15:34:09.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 5 monclient: authenticate success, global_id 4393 2023-11-02T15:34:09.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 10 monclient: _renew_subs 2023-11-02T15:34:09.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:09.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac009a70 con 0x7f3fb40a84b0 2023-11-02T15:34:09.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:09.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac03eb60 con 0x7f3fb40a84b0 2023-11-02T15:34:09.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:34:09.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3faa7fc640 10 monclient: dump: 2023-11-02T15:34:09.239 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.239 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.240 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.240 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.240 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.240 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.240 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.241 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.241 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.241 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fabfff640 4 set_mon_vals no callback set 2023-11-02T15:34:09.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 10 client.4393.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:09.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 10 monclient: _renew_subs 2023-11-02T15:34:09.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:09.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb4139c50 con 0x7f3fb40a84b0 2023-11-02T15:34:09.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.196+0000 7f3fbafa2640 1 librados: init done 2023-11-02T15:34:09.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.197+0000 7f3fbafa2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:09.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.197+0000 7f3fbafa2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:09.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.197+0000 7f3fbafa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:34:09.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.197+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb413dea0 con 0x7f3fb40a84b0 2023-11-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3fac03b050 con 0x7f3fb40a84b0 2023-11-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3faa7fc640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 0x7f3f9806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3fb979f640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 0x7f3f9806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3fb979f640 10 monclient: get_auth_request con 0x7f3f9806bdf0 auth_method 0 2023-11-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3faa7fc640 10 client.4393.objecter ms_handle_connect 0x7f3f9806bdf0 2023-11-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.434+0000 7f3fb979f640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 0x7f3f9806e2b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fb413ac60 tx=0x7f3fa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.436+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3fac07a3a0 con 0x7f3fb40a84b0 2023-11-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.436+0000 7f3faa7fc640 10 client.4393.objecter ms_dispatch 0x7f3fb4000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.436+0000 7f3faa7fc640 3 client.4393.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.436+0000 7f3faa7fc640 3 client.4393.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.436+0000 7f3faa7fc640 20 client.4393.objecter dump_active .. 0 homeless 2023-11-02T15:34:09.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.445+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3fac0766e0 con 0x7f3fb40a84b0 2023-11-02T15:34:09.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.445+0000 7f3faa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:09.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.445+0000 7f3faa7fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:09.613 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.611+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fb414be00 con 0x7f3f9806bdf0 2023-11-02T15:34:09.615 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.613+0000 7f3faa7fc640 1 -- 172.21.15.174:0/2034322264 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48061 (secure 0 0 0) 0x7f3fa400f820 con 0x7f3f9806bdf0 2023-11-02T15:34:09.616 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:09.617 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T15:34:09.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 10 librados: watch_flush enter 2023-11-02T15:34:09.620 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 10 librados: watch_flush exit 2023-11-02T15:34:09.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 20 client.4393.objecter shutdown clearing up homeless session... 2023-11-02T15:34:09.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 10 client.4393.objecter successfully canceled tick 2023-11-02T15:34:09.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 msgr2=0x7f3f9806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 0x7f3f9806e2b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fb413ac60 tx=0x7f3fa4009040 comp rx=0 tx=0).stop 2023-11-02T15:34:09.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 10 monclient: shutdown 2023-11-02T15:34:09.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:09.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 msgr2=0x7f3fb413d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 0x7f3fb413d2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fac033000 tx=0x7f3fac004a40 comp rx=0 tx=0).stop 2023-11-02T15:34:09.623 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 shutdown_connections 2023-11-02T15:34:09.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fb4139280 0x7f3fb4139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3f9806bdf0 0x7f3f9806e2b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.624 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fb40b3cd0 0x7f3fb413d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 --2- 172.21.15.174:0/2034322264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb40a84b0 0x7f3fb413d2c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 >> 172.21.15.174:0/2034322264 conn(0x7f3fb4065b40 msgr2=0x7f3fb40a8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:09.625 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 shutdown_connections 2023-11-02T15:34:09.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.615+0000 7f3fbafa2640 1 -- 172.21.15.174:0/2034322264 wait complete. 2023-11-02T15:34:09.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.616+0000 7f3fbafa2640 1 librados: shutdown 2023-11-02T15:34:09.629 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-02T15:34:08.709416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218284,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756436,"statfs":{"total":773094113280,"available":772870590464,"internally_reserved":0,"allocated":2539520,"data_stored":1393240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.962258+0000","last_change":"2023-11-02T15:34:01.938543+0000","last_active":"2023-11-02T15:34:01.962258+0000","last_peered":"2023-11-02T15:34:01.962258+0000","last_clean":"2023-11-02T15:34:01.962258+0000","last_became_active":"2023-11-02T15:34:01.933504+0000","last_became_peered":"2023-11-02T15:34:01.933504+0000","last_unstale":"2023-11-02T15:34:01.962258+0000","last_undegraded":"2023-11-02T15:34:01.962258+0000","last_fullsized":"2023-11-02T15:34:01.962258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:09:55.163100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.050836+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T15:34:02.050836+0000","last_peered":"2023-11-02T15:34:02.050836+0000","last_clean":"2023-11-02T15:34:02.050836+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T15:34:02.050836+0000","last_undegraded":"2023-11-02T15:34:02.050836+0000","last_fullsized":"2023-11-02T15:34:02.050836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.787486+0000","last_change":"2023-11-02T15:34:01.592738+0000","last_active":"2023-11-02T15:34:01.787486+0000","last_peered":"2023-11-02T15:34:01.787486+0000","last_clean":"2023-11-02T15:34:01.787486+0000","last_became_active":"2023-11-02T15:34:01.583822+0000","last_became_peered":"2023-11-02T15:34:01.583822+0000","last_unstale":"2023-11-02T15:34:01.787486+0000","last_undegraded":"2023-11-02T15:34:01.787486+0000","last_fullsized":"2023-11-02T15:34:01.787486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:23:43.663036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.024969+0000","last_change":"2023-11-02T15:34:01.994993+0000","last_active":"2023-11-02T15:34:02.024969+0000","last_peered":"2023-11-02T15:34:02.024969+0000","last_clean":"2023-11-02T15:34:02.024969+0000","last_became_active":"2023-11-02T15:34:01.966537+0000","last_became_peered":"2023-11-02T15:34:01.966537+0000","last_unstale":"2023-11-02T15:34:02.024969+0000","last_undegraded":"2023-11-02T15:34:02.024969+0000","last_fullsized":"2023-11-02T15:34:02.024969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:54:18.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.964006+0000","last_change":"2023-11-02T15:34:01.955839+0000","last_active":"2023-11-02T15:34:01.964006+0000","last_peered":"2023-11-02T15:34:01.964006+0000","last_clean":"2023-11-02T15:34:01.964006+0000","last_became_active":"2023-11-02T15:34:01.950053+0000","last_became_peered":"2023-11-02T15:34:01.950053+0000","last_unstale":"2023-11-02T15:34:01.964006+0000","last_undegraded":"2023-11-02T15:34:01.964006+0000","last_fullsized":"2023-11-02T15:34:01.964006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:08:54.230636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-02T15:34:02.533782+0000","last_change":"2023-11-02T15:34:01.430741+0000","last_active":"2023-11-02T15:34:02.533782+0000","last_peered":"2023-11-02T15:34:02.533782+0000","last_clean":"2023-11-02T15:34:02.533782+0000","last_became_active":"2023-11-02T15:34:01.403797+0000","last_became_peered":"2023-11-02T15:34:01.403797+0000","last_unstale":"2023-11-02T15:34:02.533782+0000","last_undegraded":"2023-11-02T15:34:02.533782+0000","last_fullsized":"2023-11-02T15:34:02.533782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:20:09.696671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.775554+0000","last_change":"2023-11-02T15:34:01.461505+0000","last_active":"2023-11-02T15:34:01.775554+0000","last_peered":"2023-11-02T15:34:01.775554+0000","last_clean":"2023-11-02T15:34:01.775554+0000","last_became_active":"2023-11-02T15:34:01.430918+0000","last_became_peered":"2023-11-02T15:34:01.430918+0000","last_unstale":"2023-11-02T15:34:01.775554+0000","last_undegraded":"2023-11-02T15:34:01.775554+0000","last_fullsized":"2023-11-02T15:34:01.775554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:33:03.165716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-02T15:34:01.831688+0000","last_change":"2023-11-02T15:34:01.826046+0000","last_active":"2023-11-02T15:34:01.831688+0000","last_peered":"2023-11-02T15:34:01.831688+0000","last_clean":"2023-11-02T15:34:01.831688+0000","last_became_active":"2023-11-02T15:34:01.817659+0000","last_became_peered":"2023-11-02T15:34:01.817659+0000","last_unstale":"2023-11-02T15:34:01.831688+0000","last_undegraded":"2023-11-02T15:34:01.831688+0000","last_fullsized":"2023-11-02T15:34:01.831688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:16:14.944485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"22'76","reported_seq":117,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-02T15:34:06.971301+0000","last_change":"2023-11-02T15:33:55.693910+0000","last_active":"2023-11-02T15:34:06.971301+0000","last_peered":"2023-11-02T15:34:06.971301+0000","last_clean":"2023-11-02T15:34:06.971301+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T15:34:06.971301+0000","last_undegraded":"2023-11-02T15:34:06.971301+0000","last_fullsized":"2023-11-02T15:34:06.971301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_clean_scrub_stamp":"2023-11-02T15:33:53.161912+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:29.329385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.074999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.853999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.097000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.236000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.98}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":28.859999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":618496,"data_stored":472535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.255000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8599999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.787000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.18}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.353999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.429}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0220000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.433999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.16}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.731999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.254000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.252000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5220000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0370000000000008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.723000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":618496,"data_stored":472535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.3029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.806000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.617000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.528}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.483000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.215999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.438000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1559999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.023}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.021000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.57}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":26.609000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.7640000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.838000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.510999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.002000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.446000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.530999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":74695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6509999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.625999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.602}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.577}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.864999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T15:34:09.633 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-02T15:34:09.633 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-02T15:34:09.633 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-02T15:34:09.634 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-02T15:34:09.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.730+0000 7f733142d640 10 monclient: get_monmap_and_config 2023-11-02T15:34:09.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.730+0000 7f733142d640 10 monclient: build_initial_monmap 2023-11-02T15:34:09.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.730+0000 7f733142d640 10 monclient: monmap: 2023-11-02T15:34:09.732 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:09.733 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.733 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:34:09.731383+0000 2023-11-02T15:34:09.733 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:34:09.731383+0000 2023-11-02T15:34:09.733 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:09.733 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.734 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T15:34:09.734 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T15:34:09.734 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T15:34:09.734 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T15:34:09.735 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T15:34:09.735 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 Processor -- start 2023-11-02T15:34:09.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 -- start start 2023-11-02T15:34:09.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient: init 2023-11-02T15:34:09.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:09.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient: _add_conns ranks=[0,4,2,3,1] 2023-11-02T15:34:09.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f732c0b4170 addr v1:172.21.15.174:6789/0 2023-11-02T15:34:09.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f732c0a8df0 0x7f732c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): picked mon.noname-c con 0x7f732c0a8df0 addr v2:172.21.15.175:3300/0 2023-11-02T15:34:09.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 0x7f732c13e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): picked mon.noname-b con 0x7f732c0b13e0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:09.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0aa3d0 con 0x7f732c0b4170 2023-11-02T15:34:09.740 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c13f420 con 0x7f732c0a8df0 2023-11-02T15:34:09.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0b4550 con 0x7f732c0b13e0 2023-11-02T15:34:09.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:09.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.731+0000 7f733142d640 10 monclient(hunting): authenticate will time out at 1377.073484s 2023-11-02T15:34:09.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732affd640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f732c0b4170 0x7f732c141800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:51382/0 (socket says 172.21.15.174:51382) 2023-11-02T15:34:09.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732affd640 1 -- 172.21.15.174:0/1129329967 learned_addr learned my addr 172.21.15.174:0/1129329967 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T15:34:09.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732b7fe640 1 --2- 172.21.15.174:0/1129329967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 0x7f732c13e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f7323fff640 1 --2- 172.21.15.174:0/1129329967 >> v2:172.21.15.175:3300/0 conn(0x7f732c0a8df0 0x7f732c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732b7fe640 10 monclient(hunting): get_auth_request con 0x7f732c0b13e0 auth_method 0 2023-11-02T15:34:09.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f732b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f7323fff640 10 monclient(hunting): get_auth_request con 0x7f732c0a8df0 auth_method 0 2023-11-02T15:34:09.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f7323fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f7323fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.732+0000 7f7323fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.733+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 1 -- 172.21.15.174:0/1129329967 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501509798 0 0) 0x7f732c0aa3d0 con 0x7f732c0b4170 2023-11-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 10 monclient(hunting): _negotiate 2023-11-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 10 monclient(hunting): my global_id is 4329 2023-11-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732a7fc640 1 -- 172.21.15.174:0/1129329967 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7318003520 con 0x7f732c0b4170 2023-11-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 10 monclient(hunting): handle_auth_done global_id 4328 payload 274 2023-11-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 1 -- 172.21.15.174:0/1129329967 >> v2:172.21.15.175:3300/0 conn(0x7f732c0a8df0 msgr2=0x7f732c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 1 --2- 172.21.15.174:0/1129329967 >> v2:172.21.15.175:3300/0 conn(0x7f732c0a8df0 0x7f732c141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 1 -- 172.21.15.174:0/1129329967 >> v1:172.21.15.174:6789/0 conn(0x7f732c0b4170 legacy=0x7f732c141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 10 monclient: _finish_hunting 0 2023-11-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 1 monclient: found mon.noname-b 2023-11-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.734+0000 7f732b7fe640 1 -- 172.21.15.174:0/1129329967 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0a92d0 con 0x7f732c0b13e0 2023-11-02T15:34:09.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.736+0000 7f732b7fe640 1 --2- 172.21.15.174:0/1129329967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 0x7f732c13e340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7324009dc0 tx=0x7f732402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816f090dbbbcd6b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 1 -- 172.21.15.174:0/1129329967 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732402cbb0 con 0x7f732c0b13e0 2023-11-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 10 monclient: dump: 2023-11-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.762 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.762 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.762 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.762 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 10 monclient: _finish_auth 0 2023-11-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 10 monclient: _check_auth_tickets 2023-11-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f732a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f733142d640 5 monclient: authenticate success, global_id 4328 2023-11-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.738+0000 7f733142d640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 1 -- 172.21.15.174:0/1129329967 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732402cd10 con 0x7f732c0b13e0 2023-11-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 1 -- 172.21.15.174:0/1129329967 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324034640 con 0x7f732c0b13e0 2023-11-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f733142d640 10 monclient: get_monmap_and_config success 2023-11-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f733142d640 4 set_mon_vals no callback set 2023-11-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.740+0000 7f732a7fc640 10 monclient: dump: 2023-11-02T15:34:09.766 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.766 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.766 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.766 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.766 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.767 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.767 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.767 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.767 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.767 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 10 monclient: shutdown 2023-11-02T15:34:09.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:09.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 -- 172.21.15.174:0/1129329967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 msgr2=0x7f732c13e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 --2- 172.21.15.174:0/1129329967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 0x7f732c13e340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7324009dc0 tx=0x7f732402be00 comp rx=0 tx=0).stop 2023-11-02T15:34:09.768 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 -- 172.21.15.174:0/1129329967 shutdown_connections 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 --2- 172.21.15.174:0/1129329967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0b13e0 0x7f732c13e340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 --2- 172.21.15.174:0/1129329967 >> v2:172.21.15.175:3300/0 conn(0x7f732c0a8df0 0x7f732c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 -- 172.21.15.174:0/1129329967 >> 172.21.15.174:0/1129329967 conn(0x7f732c065d80 msgr2=0x7f732c066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 -- 172.21.15.174:0/1129329967 shutdown_connections 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 1 -- 172.21.15.174:0/1129329967 wait complete. 2023-11-02T15:34:09.769 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 10 monclient: build_initial_monmap 2023-11-02T15:34:09.770 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.741+0000 7f733142d640 10 monclient: monmap: 2023-11-02T15:34:09.770 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T15:34:09.770 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:34:09.770 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T15:34:09.771 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T15:34:09.772 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T15:34:09.772 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.773 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:34:09.773 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:34:09.773 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:34:09.774 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 librados: starting msgr at 2023-11-02T15:34:09.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 librados: starting objecter 2023-11-02T15:34:09.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 Processor -- start 2023-11-02T15:34:09.774 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 -- start start 2023-11-02T15:34:09.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 librados: setting wanted keys 2023-11-02T15:34:09.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 librados: calling monclient init 2023-11-02T15:34:09.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient: init 2023-11-02T15:34:09.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient: _reopen_session rank -1 2023-11-02T15:34:09.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:34:09.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): picked mon.noname-c con 0x7f732c0a8df0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:34:09.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732c0b13e0 0x7f732c0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): picked mon.noname-b con 0x7f732c0b13e0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:34:09.777 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c0b4170 0x7f732c1d6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.777 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): picked mon.noname-a con 0x7f732c0b4170 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:34:09.778 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.778 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0addd0 con 0x7f732c0b4170 2023-11-02T15:34:09.778 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.778 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0ac150 con 0x7f732c0b13e0 2023-11-02T15:34:09.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): start opening mon connection 2023-11-02T15:34:09.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0ac2c0 con 0x7f732c0a8df0 2023-11-02T15:34:09.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): _renew_subs 2023-11-02T15:34:09.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.742+0000 7f733142d640 10 monclient(hunting): authenticate will time out at 1377.084613s 2023-11-02T15:34:09.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c0b4170 0x7f732c1d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:58154/0 (socket says 172.21.15.174:58154) 2023-11-02T15:34:09.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 1 -- 172.21.15.174:0/2197923792 learned_addr learned my addr 172.21.15.174:0/2197923792 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 10 monclient(hunting): get_auth_request con 0x7f732c0a8df0 auth_method 0 2023-11-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732affd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732b7fe640 10 monclient(hunting): get_auth_request con 0x7f732c0b4170 auth_method 0 2023-11-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732b7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f732b7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.743+0000 7f7323fff640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732c0b13e0 0x7f732c0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): get_auth_request con 0x7f732c0b13e0 auth_method 0 2023-11-02T15:34:09.784 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f732b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.744+0000 7f7323fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 10 monclient(hunting): handle_auth_done global_id 4331 payload 824 2023-11-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 -- 172.21.15.174:0/2197923792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732c0b13e0 msgr2=0x7f732c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732c0b13e0 0x7f732c0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 -- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c0b4170 msgr2=0x7f732c1d6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:09.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c0b4170 0x7f732c1d6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:09.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 10 monclient: _finish_hunting 0 2023-11-02T15:34:09.788 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 monclient: found mon.noname-c 2023-11-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.746+0000 7f732affd640 1 -- 172.21.15.174:0/2197923792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324009a70 con 0x7f732c0a8df0 2023-11-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.749+0000 7f732affd640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c390 tx=0x7f731c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.749+0000 7f73237fe640 10 client.?.objecter ms_handle_connect 0x7f732c0a8df0 2023-11-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.749+0000 7f73237fe640 10 client.?.objecter resend_mon_ops 2023-11-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c005ce0 con 0x7f732c0a8df0 2023-11-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: dump: 2023-11-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: _finish_auth 0 2023-11-02T15:34:09.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: _check_auth_tickets 2023-11-02T15:34:09.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:34:09.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c005e40 con 0x7f732c0a8df0 2023-11-02T15:34:09.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 5 monclient: authenticate success, global_id 4331 2023-11-02T15:34:09.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 10 monclient: _renew_subs 2023-11-02T15:34:09.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0ada00 con 0x7f732c0a8df0 2023-11-02T15:34:09.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:34:09.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0072f0 con 0x7f732c0a8df0 2023-11-02T15:34:09.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:34:09.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T15:34:09.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f73237fe640 10 monclient: dump: 2023-11-02T15:34:09.802 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T15:34:09.802 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:34:09.802 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.803 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:34:09.803 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T15:34:09.803 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T15:34:09.803 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:34:09.804 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:34:09.804 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:34:09.804 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T15:34:09.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f7329ffb640 4 set_mon_vals no callback set 2023-11-02T15:34:09.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 10 client.4331.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:34:09.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 10 monclient: _renew_subs 2023-11-02T15:34:09.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c1d6bb0 con 0x7f732c0a8df0 2023-11-02T15:34:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.752+0000 7f733142d640 1 librados: init done 2023-11-02T15:34:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.753+0000 7f733142d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.753+0000 7f733142d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.753+0000 7f733142d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.753+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c04e040 con 0x7f732c0a8df0 2023-11-02T15:34:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.771+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f731c007450 con 0x7f732c0a8df0 2023-11-02T15:34:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.771+0000 7f73237fe640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 0x7f730c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:34:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.772+0000 7f7323fff640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 0x7f730c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:34:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.772+0000 7f7323fff640 10 monclient: get_auth_request con 0x7f730c06be10 auth_method 0 2023-11-02T15:34:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.772+0000 7f7323fff640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 0x7f730c06e2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f732c0ac810 tx=0x7f7314009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:34:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.772+0000 7f73237fe640 10 client.4331.objecter ms_handle_connect 0x7f730c06be10 2023-11-02T15:34:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.774+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f731c057660 con 0x7f732c0a8df0 2023-11-02T15:34:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.774+0000 7f73237fe640 10 client.4331.objecter ms_dispatch 0x7f732c000b90 osd_map(22..22 src has 1..22) v4 2023-11-02T15:34:09.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.774+0000 7f73237fe640 3 client.4331.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:34:09.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.774+0000 7f73237fe640 3 client.4331.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:34:09.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:09.774+0000 7f73237fe640 20 client.4331.objecter dump_active .. 0 homeless 2023-11-02T15:34:10.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.007+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f731c0539a0 con 0x7f732c0a8df0 2023-11-02T15:34:10.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.007+0000 7f73237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T15:34:10.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.007+0000 7f73237fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T15:34:10.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.256+0000 7f733142d640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2023-11-02T15:34:10.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.256+0000 7f733142d640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2023-11-02T15:34:10.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.256+0000 7f733142d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T15:34:10.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.256+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f732c0a84b0 con 0x7f732c0a8df0 2023-11-02T15:34:10.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.263+0000 7f73237fe640 1 -- 172.21.15.174:0/2197923792 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f731c017080 con 0x7f732c0a8df0 2023-11-02T15:34:10.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.263+0000 7f73237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2023-11-02T15:34:10.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.263+0000 7f73237fe640 10 monclient: _finish_command 2 = system:0 2023-11-02T15:34:10.265 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T15:34:10.265 INFO:teuthology.orchestra.run.smithi174.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-02T15:34:10.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 10 librados: watch_flush enter 2023-11-02T15:34:10.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 10 librados: watch_flush exit 2023-11-02T15:34:10.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 20 client.4331.objecter shutdown clearing up homeless session... 2023-11-02T15:34:10.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 10 client.4331.objecter successfully canceled tick 2023-11-02T15:34:10.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 msgr2=0x7f730c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:10.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 0x7f730c06e2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f732c0ac810 tx=0x7f7314009190 comp rx=0 tx=0).stop 2023-11-02T15:34:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 10 monclient: shutdown 2023-11-02T15:34:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:34:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 msgr2=0x7f732c1456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:34:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c390 tx=0x7f731c00c850 comp rx=0 tx=0).stop 2023-11-02T15:34:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 shutdown_connections 2023-11-02T15:34:10.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c0b4170 0x7f732c1d6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:10.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732c0b13e0 0x7f732c0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:10.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f730c06be10 0x7f730c06e2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:10.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 --2- 172.21.15.174:0/2197923792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0a8df0 0x7f732c1456a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:34:10.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.265+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 >> 172.21.15.174:0/2197923792 conn(0x7f732c065d80 msgr2=0x7f732c0b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:34:10.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.266+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 shutdown_connections 2023-11-02T15:34:10.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.266+0000 7f733142d640 1 -- 172.21.15.174:0/2197923792 wait complete. 2023-11-02T15:34:10.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T15:34:10.266+0000 7f733142d640 1 librados: shutdown 2023-11-02T15:34:10.278 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-02T15:34:10.279 INFO:teuthology.run_tasks:Running task workunit... 2023-11-02T15:34:10.292 INFO:tasks.workunit:Pulling workunits from ref b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:34:10.293 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-02T15:34:10.294 DEBUG:teuthology.orchestra.run.smithi174:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-02T15:34:10.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T15:34:10.313 INFO:teuthology.orchestra.run.smithi174.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-02T15:34:10.313 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-11-02T15:34:10.368 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-11-02T15:34:10.369 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-11-02T15:34:10.425 INFO:tasks.workunit:timeout=6h 2023-11-02T15:34:10.426 INFO:tasks.workunit:cleanup=True 2023-11-02T15:34:10.426 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b5a9486b4a6f144f83c51b7ec33111a2c86dc081 2023-11-02T15:34:10.483 INFO:tasks.workunit.client.0.smithi174.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-02T15:34:22.936 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:35.089 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:22.962 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:35.115 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:22.988 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:35.084 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:22.993 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:35.090 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:22.996 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:35.150 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:23.001 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:35.155 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:23.009 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:35.162 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:23.030 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:35.126 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:23.046 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:35.200 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:42.949 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:55.102 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:42.998 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:55.094 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:42.999 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:55.096 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.014 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:01:55.110 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.027 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:55.181 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:01:55.231 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.119 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:55.272 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.133 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:55.287 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:34:43.153 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:01:55.306 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.058 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:15.212 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.062 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:15.215 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.075 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:15.229 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.112 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:15.263 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.112 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:15.206 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.112 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:15.208 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.113 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:15.266 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.126 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:15.222 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:03.129 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:15.282 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:05.497 INFO:tasks.workunit.client.0.smithi174.stderr:Note: switching to 'b5a9486b4a6f144f83c51b7ec33111a2c86dc081'. 2023-11-02T15:35:05.498 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.498 INFO:tasks.workunit.client.0.smithi174.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-02T15:35:05.498 INFO:tasks.workunit.client.0.smithi174.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-02T15:35:05.498 INFO:tasks.workunit.client.0.smithi174.stderr:state without impacting any branches by switching back to a branch. 2023-11-02T15:35:05.499 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.499 INFO:tasks.workunit.client.0.smithi174.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-02T15:35:05.499 INFO:tasks.workunit.client.0.smithi174.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-02T15:35:05.499 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.499 INFO:tasks.workunit.client.0.smithi174.stderr: git switch -c 2023-11-02T15:35:05.500 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.500 INFO:tasks.workunit.client.0.smithi174.stderr:Or undo this operation with: 2023-11-02T15:35:05.500 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.500 INFO:tasks.workunit.client.0.smithi174.stderr: git switch - 2023-11-02T15:35:05.500 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.501 INFO:tasks.workunit.client.0.smithi174.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-02T15:35:05.501 INFO:tasks.workunit.client.0.smithi174.stderr: 2023-11-02T15:35:05.501 INFO:tasks.workunit.client.0.smithi174.stderr:HEAD is now at b5a9486b4a6 Merge branch 'wip-63387-reef' of https://github.com/ajarr/ceph into wip-yuri6-testing-2023-11-01-0745-reef 2023-11-02T15:35:05.502 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-02T15:35:05.563 INFO:tasks.workunit.client.0.smithi174.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-02T15:35:05.564 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-02T15:35:05.565 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-02T15:35:06.186 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-02T15:35:06.257 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-02T15:35:06.297 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-02T15:35:06.302 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-02T15:35:06.303 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-02T15:35:06.337 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-02T15:35:06.340 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-02T15:35:06.340 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-02T15:35:06.397 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2023-11-02T15:35:06.398 INFO:tasks.workunit:Running workunit rados/test.sh... 2023-11-02T15:35:06.399 DEBUG:teuthology.orchestra.run.smithi174:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b5a9486b4a6f144f83c51b7ec33111a2c86dc081 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2023-11-02T15:35:06.457 INFO:tasks.workunit.client.0.smithi174.stderr:+ parallel=1 2023-11-02T15:35:06.457 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' '' = --serial ']' 2023-11-02T15:35:06.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ color= 2023-11-02T15:35:06.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -t 1 ']' 2023-11-02T15:35:06.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2023-11-02T15:35:06.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ declare -A pids 2023-11-02T15:35:06.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.459 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.459 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_aio 2023-11-02T15:35:06.459 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_aio' 2023-11-02T15:35:06.459 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_aio 2023-11-02T15:35:06.459 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.460 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_aio 2023-11-02T15:35:06.460 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91302 2023-11-02T15:35:06.461 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_aio on pid 91302' 2023-11-02T15:35:06.461 INFO:tasks.workunit.client.0.smithi174.stdout:test api_aio on pid 91302 2023-11-02T15:35:06.475 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91302 2023-11-02T15:35:06.475 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.476 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.476 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2023-11-02T15:35:06.476 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_aio_pp 2023-11-02T15:35:06.477 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_aio_pp' 2023-11-02T15:35:06.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_aio_pp 2023-11-02T15:35:06.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.478 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.478 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.478 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.478 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.480 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.480 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.480 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.480 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.480 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.481 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.483 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.485 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.485 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.485 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_aio_pp 2023-11-02T15:35:06.485 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91309 2023-11-02T15:35:06.485 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_aio_pp on pid 91309' 2023-11-02T15:35:06.486 INFO:tasks.workunit.client.0.smithi174.stdout:test api_aio_pp on pid 91309 2023-11-02T15:35:06.501 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91309 2023-11-02T15:35:06.501 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.501 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.501 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2023-11-02T15:35:06.502 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_io 2023-11-02T15:35:06.502 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_io' 2023-11-02T15:35:06.502 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_io 2023-11-02T15:35:06.503 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.503 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.503 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.503 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.503 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.504 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.504 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.504 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.506 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.506 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.506 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.507 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.511 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.513 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.514 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.514 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.514 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.514 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.515 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.515 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.515 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.517 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.517 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.518 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.518 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.518 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.518 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.518 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.520 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.520 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.521 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.521 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_io 2023-11-02T15:35:06.522 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91318 2023-11-02T15:35:06.523 INFO:tasks.workunit.client.0.smithi174.stdout:test api_io on pid 91318 2023-11-02T15:35:06.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_io on pid 91318' 2023-11-02T15:35:06.535 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91318 2023-11-02T15:35:06.535 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.535 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.535 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2023-11-02T15:35:06.536 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_io_pp 2023-11-02T15:35:06.536 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_io_pp' 2023-11-02T15:35:06.536 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_io_pp 2023-11-02T15:35:06.536 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.537 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.538 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.538 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.538 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.539 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.539 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.539 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.539 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.539 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.540 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.541 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.541 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.541 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.541 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.541 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.543 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.543 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.543 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.543 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.543 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.544 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.544 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.544 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.544 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.545 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.545 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.545 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.545 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.545 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.546 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.546 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.546 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.546 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.546 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.547 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_io_pp 2023-11-02T15:35:06.548 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91329 2023-11-02T15:35:06.548 INFO:tasks.workunit.client.0.smithi174.stdout:test api_io_pp on pid 91329 2023-11-02T15:35:06.554 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_io_pp on pid 91329' 2023-11-02T15:35:06.554 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91329 2023-11-02T15:35:06.554 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.554 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.555 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2023-11-02T15:35:06.555 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_asio 2023-11-02T15:35:06.555 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_asio' 2023-11-02T15:35:06.555 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_asio 2023-11-02T15:35:06.555 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.557 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.557 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.558 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.559 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.563 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.563 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.564 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.564 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.564 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.564 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.564 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.565 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.565 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.565 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.566 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.566 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.566 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.566 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.567 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.567 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.567 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.567 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.567 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.568 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.568 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.568 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.568 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.568 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.569 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.569 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.569 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.569 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.570 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.570 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.570 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.570 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.571 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.571 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.571 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.571 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.571 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.573 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.578 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.578 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.578 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.578 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.579 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.579 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.579 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.580 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.580 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.580 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.580 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.580 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.581 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.581 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_asio 2023-11-02T15:35:06.581 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91343 2023-11-02T15:35:06.581 INFO:tasks.workunit.client.0.smithi174.stdout:test api_asio on pid 91343 2023-11-02T15:35:06.591 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_asio on pid 91343' 2023-11-02T15:35:06.591 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91343 2023-11-02T15:35:06.592 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.592 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.592 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2023-11-02T15:35:06.592 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_list 2023-11-02T15:35:06.593 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_list' 2023-11-02T15:35:06.593 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.595 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.596 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_list 2023-11-02T15:35:06.596 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.597 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.597 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.597 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.597 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.598 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.598 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.598 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.599 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.600 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.601 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.601 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.601 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.601 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.602 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.602 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.602 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.603 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.603 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.603 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.604 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.604 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.604 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.605 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.606 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.607 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.607 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.607 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.608 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.608 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.608 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.609 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.609 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.609 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.609 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.610 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.610 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.610 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.610 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.610 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.611 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.611 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.611 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.611 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.612 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.612 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.612 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.612 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.613 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.613 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.613 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.613 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.614 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.614 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.615 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.615 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.616 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.616 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.616 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.617 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.618 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.618 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.618 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.618 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.619 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.620 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.620 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.620 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.620 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.621 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.621 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.621 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.621 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.622 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.622 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.622 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.623 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.623 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.623 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.623 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.624 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.624 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.624 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.624 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.627 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.628 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.628 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.628 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.629 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_list 2023-11-02T15:35:06.629 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91302/exe 2023-11-02T15:35:06.629 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.629 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.630 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91362 2023-11-02T15:35:06.630 INFO:tasks.workunit.client.0.smithi174.stdout:test api_list on pid 91362 2023-11-02T15:35:06.645 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_list on pid 91362' 2023-11-02T15:35:06.646 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91362 2023-11-02T15:35:06.646 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.646 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.646 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.647 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2023-11-02T15:35:06.647 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.647 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.648 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_lock 2023-11-02T15:35:06.649 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.649 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_lock' 2023-11-02T15:35:06.650 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:06.650 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_lock 2023-11-02T15:35:06.651 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.651 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.658 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.659 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.659 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.659 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.660 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.660 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.662 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.664 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.664 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.664 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.665 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.665 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.665 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.665 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.666 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.666 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.666 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.666 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.667 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.667 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.668 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.668 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.669 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.669 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.675 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.675 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.676 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.676 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.676 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.677 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.677 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.677 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.678 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.678 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.678 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.678 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.679 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.680 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:06.685 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.685 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:06.685 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.685 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.686 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.686 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.686 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:06.686 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.687 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.687 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:06.687 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.687 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.687 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.688 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.688 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.688 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:06.688 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.689 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:06.689 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.689 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.689 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.689 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.690 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.690 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:06.690 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.690 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.691 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.691 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:06.691 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.691 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.692 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:06.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.693 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:06.693 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.693 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.693 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.694 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.694 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.695 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.695 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.695 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.695 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.696 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.696 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.696 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.697 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.697 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_aio 2023-11-02T15:35:06.697 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.697 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.699 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.700 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.700 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_aio.log 2023-11-02T15:35:06.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.701 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.701 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.701 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91309/exe 2023-11-02T15:35:06.701 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_aio: /' 2023-11-02T15:35:06.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.703 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:06.703 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_lock 2023-11-02T15:35:06.703 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.703 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91383 2023-11-02T15:35:06.704 INFO:tasks.workunit.client.0.smithi174.stdout:test api_lock on pid 91383 2023-11-02T15:35:06.712 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_lock on pid 91383' 2023-11-02T15:35:06.713 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91383 2023-11-02T15:35:06.713 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.713 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.714 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2023-11-02T15:35:06.714 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.714 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.715 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.715 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.715 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.715 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.716 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.716 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_lock_pp 2023-11-02T15:35:06.716 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.718 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.718 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_lock_pp' 2023-11-02T15:35:06.718 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:06.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:06.719 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.719 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.720 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.720 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:06.720 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.721 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:06.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:06.721 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.721 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.721 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.722 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.722 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:06.722 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.722 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:06.723 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.723 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.723 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.723 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:06.724 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.724 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.724 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:06.724 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:06.724 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_lock_pp 2023-11-02T15:35:06.725 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.725 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.725 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.726 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_aio_pp.log 2023-11-02T15:35:06.726 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_aio_pp 2023-11-02T15:35:06.726 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_aio_pp: /' 2023-11-02T15:35:06.726 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.726 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.727 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.727 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.727 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.727 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.728 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.728 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.728 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.729 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.729 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.729 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.729 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.729 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.730 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.730 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.730 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.730 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.731 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.731 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.731 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.731 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.731 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.732 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.732 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.732 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.732 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.733 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.734 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.734 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.735 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.735 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.735 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.736 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.736 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.736 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.736 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.737 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.737 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.737 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.738 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.738 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_lock_pp 2023-11-02T15:35:06.738 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.738 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.739 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.739 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.739 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.739 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91398 2023-11-02T15:35:06.740 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.740 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.741 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_lock_pp on pid 91398' 2023-11-02T15:35:06.741 INFO:tasks.workunit.client.0.smithi174.stdout:test api_lock_pp on pid 91398 2023-11-02T15:35:06.760 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.760 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91398 2023-11-02T15:35:06.760 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2023-11-02T15:35:06.760 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.761 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.761 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_misc 2023-11-02T15:35:06.761 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.761 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.762 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.763 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.763 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_misc' 2023-11-02T15:35:06.763 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.763 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91318/exe 2023-11-02T15:35:06.764 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.767 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.767 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.767 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.768 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.768 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.768 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.768 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.768 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.769 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.769 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.770 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.770 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.770 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.770 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.771 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_misc 2023-11-02T15:35:06.773 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.774 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.774 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:06.774 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.774 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.776 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.776 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.776 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.776 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:06.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:06.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:06.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:06.780 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.781 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.781 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:06.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:06.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.783 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.783 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:06.783 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.783 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.783 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.784 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.784 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.784 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:06.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:06.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.786 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:06.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.787 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.789 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.789 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.790 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.790 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.791 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.791 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_io 2023-11-02T15:35:06.791 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_io: /' 2023-11-02T15:35:06.799 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.802 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.802 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.811 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.813 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_misc 2023-11-02T15:35:06.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.816 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.816 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91420 2023-11-02T15:35:06.816 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.816 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.817 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_misc on pid 91420' 2023-11-02T15:35:06.817 INFO:tasks.workunit.client.0.smithi174.stdout:test api_misc on pid 91420 2023-11-02T15:35:06.830 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91420 2023-11-02T15:35:06.830 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.830 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.831 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_misc_pp 2023-11-02T15:35:06.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_misc_pp' 2023-11-02T15:35:06.831 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:06.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:06.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:06.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:06.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.835 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:06.835 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:06.835 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.836 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:06.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:06.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.839 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:06.839 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.839 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:06.839 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:06.840 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.840 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:06.840 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.840 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.841 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.841 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:06.841 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.841 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.841 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.842 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_misc_pp 2023-11-02T15:35:06.842 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.842 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2023-11-02T15:35:06.842 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.842 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.843 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.845 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.845 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_io.log 2023-11-02T15:35:06.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91343/exe 2023-11-02T15:35:06.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.865 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.866 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.867 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:06.867 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.867 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:06.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:06.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:06.868 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.868 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:06.870 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.871 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:06.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:06.871 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.872 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:06.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:06.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:06.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:06.874 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:06.874 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:06.875 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:06.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.881 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:06.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:06.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:06.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:06.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_misc_pp 2023-11-02T15:35:06.883 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_asio: /' 2023-11-02T15:35:06.883 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91442 2023-11-02T15:35:06.883 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_misc_pp on pid 91442' 2023-11-02T15:35:06.883 INFO:tasks.workunit.client.0.smithi174.stdout:test api_misc_pp on pid 91442 2023-11-02T15:35:06.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91442 2023-11-02T15:35:06.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:06.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:06.893 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_tier_pp 2023-11-02T15:35:06.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:06.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:06.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:06.895 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:06.896 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_asio 2023-11-02T15:35:06.896 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:06.901 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:06.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:06.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:06.902 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:06.902 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:06.902 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.902 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:06.903 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_tier_pp' 2023-11-02T15:35:06.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:06.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.907 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:06.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.915 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:06.915 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.916 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.916 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:06.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:06.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:06.924 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_asio.log 2023-11-02T15:35:06.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:06.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:06.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:06.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:06.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:06.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:06.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:06.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:06.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:06.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:06.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:06.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:06.928 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:06.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91329/exe 2023-11-02T15:35:06.929 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:06.929 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2023-11-02T15:35:06.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:06.929 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:06.929 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:06.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:06.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:06.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:06.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_tier_pp 2023-11-02T15:35:06.931 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:06.931 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:06.931 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:06.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:06.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:06.935 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:06.935 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:06.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:06.936 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:06.938 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:06.938 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:06.939 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:06.939 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_io_pp 2023-11-02T15:35:06.939 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_tier_pp 2023-11-02T15:35:06.939 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_io_pp: /' 2023-11-02T15:35:06.947 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91458 2023-11-02T15:35:06.947 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:06.947 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:06.948 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:06.948 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:06.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:06.949 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_tier_pp on pid 91458' 2023-11-02T15:35:06.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:06.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:06.950 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:06.950 INFO:tasks.workunit.client.0.smithi174.stdout:test api_tier_pp on pid 91458 2023-11-02T15:35:07.021 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.022 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91458 2023-11-02T15:35:07.022 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.022 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.023 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.023 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.023 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.023 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.023 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.024 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.024 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.024 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.024 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.025 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2023-11-02T15:35:07.026 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.026 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.026 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_io_pp.log 2023-11-02T15:35:07.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.028 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.033 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.033 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.034 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.034 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.034 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.034 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.034 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.035 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.036 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.036 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.036 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.036 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.036 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91383/exe 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.037 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.038 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.038 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.038 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.038 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.039 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.041 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.045 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.045 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.045 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.045 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.045 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.046 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.046 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.046 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.046 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.047 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.047 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.047 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_lock: /' 2023-11-02T15:35:07.047 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.047 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.048 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.048 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.048 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.048 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.048 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.050 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.050 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.051 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91362/exe 2023-11-02T15:35:07.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.057 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.058 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.058 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.058 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.058 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.059 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.059 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.059 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.059 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.059 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.062 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.062 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_lock.log 2023-11-02T15:35:07.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.063 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.063 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.065 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.065 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.066 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.066 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.067 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.068 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_lock 2023-11-02T15:35:07.068 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.068 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.069 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.073 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.073 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.073 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_list.log 2023-11-02T15:35:07.073 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_list: /' 2023-11-02T15:35:07.074 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_pool 2023-11-02T15:35:07.074 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_pool' 2023-11-02T15:35:07.074 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.074 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.074 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_pool 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.075 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.076 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.076 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.080 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.080 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.080 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.080 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.080 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_list 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.081 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91398/exe 2023-11-02T15:35:07.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.086 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.087 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.087 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.087 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.087 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.088 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.088 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.088 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.088 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.089 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.089 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.089 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.089 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.090 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.090 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.090 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.090 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.090 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.091 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_lock_pp.log 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.095 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.096 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.096 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.096 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.096 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.096 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.097 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_pool 2023-11-02T15:35:07.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.097 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91520 2023-11-02T15:35:07.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.098 INFO:tasks.workunit.client.0.smithi174.stdout:test api_pool on pid 91520 2023-11-02T15:35:07.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_pool on pid 91520' 2023-11-02T15:35:07.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91520 2023-11-02T15:35:07.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_lock_pp 2023-11-02T15:35:07.113 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.113 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.114 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.114 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.114 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.114 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.119 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.119 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.119 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_lock_pp: /' 2023-11-02T15:35:07.120 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91442/exe 2023-11-02T15:35:07.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.124 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91420/exe 2023-11-02T15:35:07.124 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2023-11-02T15:35:07.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_snapshots 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_snapshots' 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.125 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.126 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.126 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.126 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.126 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.127 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.127 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.128 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_misc_pp: /' 2023-11-02T15:35:07.128 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_misc_pp 2023-11-02T15:35:07.128 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_misc_pp.log 2023-11-02T15:35:07.128 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.129 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.130 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.130 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.131 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.132 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.132 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.132 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.132 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.132 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.133 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.133 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.133 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.133 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.133 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.134 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.134 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91458/exe 2023-11-02T15:35:07.134 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.134 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.134 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.135 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.137 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.137 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.137 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.137 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_snapshots 2023-11-02T15:35:07.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_tier_pp 2023-11-02T15:35:07.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_tier_pp.log 2023-11-02T15:35:07.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_tier_pp: /' 2023-11-02T15:35:07.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_misc: /' 2023-11-02T15:35:07.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_misc.log 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_misc 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.139 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.140 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.141 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.143 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_snapshots 2023-11-02T15:35:07.143 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91593 2023-11-02T15:35:07.144 INFO:tasks.workunit.client.0.smithi174.stdout:test api_snapshots on pid 91593 2023-11-02T15:35:07.154 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_snapshots on pid 91593' 2023-11-02T15:35:07.155 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91593 2023-11-02T15:35:07.155 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.155 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.155 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.155 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.156 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.156 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.156 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.156 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.156 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.157 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.157 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.157 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.157 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.157 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.158 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.158 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.158 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.159 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.159 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.159 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.159 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.159 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_snapshots_pp 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_snapshots_pp' 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_snapshots_pp 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.165 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.170 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.170 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.172 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_snapshots_pp 2023-11-02T15:35:07.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.174 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.174 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.175 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.176 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.176 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.176 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.176 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.176 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.177 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.177 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.177 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91627 2023-11-02T15:35:07.177 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_snapshots_pp on pid 91627' 2023-11-02T15:35:07.178 INFO:tasks.workunit.client.0.smithi174.stdout:test api_snapshots_pp on pid 91627 2023-11-02T15:35:07.192 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91627 2023-11-02T15:35:07.192 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.192 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.194 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.194 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2023-11-02T15:35:07.194 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.195 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.196 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.196 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.196 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.196 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.196 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.197 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.197 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.197 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.197 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.197 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.198 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.198 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.198 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.199 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.200 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.200 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.202 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.202 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.202 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.202 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.202 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.203 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.203 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.203 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.203 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.203 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_stat 2023-11-02T15:35:07.205 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_stat' 2023-11-02T15:35:07.206 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.206 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.206 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.206 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_stat 2023-11-02T15:35:07.206 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.209 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.209 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.209 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.209 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.211 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.212 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.212 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.212 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.212 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.212 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.213 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.216 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91593/exe 2023-11-02T15:35:07.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.216 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.216 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.218 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.218 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.218 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.218 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.220 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.220 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_stat 2023-11-02T15:35:07.221 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.222 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.222 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.222 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.222 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.223 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91667 2023-11-02T15:35:07.224 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_stat on pid 91667' 2023-11-02T15:35:07.224 INFO:tasks.workunit.client.0.smithi174.stdout:test api_stat on pid 91667 2023-11-02T15:35:07.229 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91667 2023-11-02T15:35:07.229 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.229 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.230 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.230 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.230 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.230 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_stat_pp 2023-11-02T15:35:07.231 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_snapshots 2023-11-02T15:35:07.231 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_stat_pp' 2023-11-02T15:35:07.231 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91520/exe 2023-11-02T15:35:07.231 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2023-11-02T15:35:07.231 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_snapshots.log 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.233 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_pool: /' 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.238 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_snapshots: /' 2023-11-02T15:35:07.238 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.238 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_pool.log 2023-11-02T15:35:07.238 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_pool 2023-11-02T15:35:07.238 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_stat_pp 2023-11-02T15:35:07.239 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_stat_pp 2023-11-02T15:35:07.239 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91704 2023-11-02T15:35:07.239 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_stat_pp on pid 91704' 2023-11-02T15:35:07.239 INFO:tasks.workunit.client.0.smithi174.stdout:test api_stat_pp on pid 91704 2023-11-02T15:35:07.241 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91704 2023-11-02T15:35:07.241 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.242 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.244 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_watch_notify 2023-11-02T15:35:07.244 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.244 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.244 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_watch_notify' 2023-11-02T15:35:07.245 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.245 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.245 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.245 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.245 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2023-11-02T15:35:07.246 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2023-11-02T15:35:07.247 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] Global test environment set-up. 2023-11-02T15:35:07.247 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] 12 tests from AsioRados 2023-11-02T15:35:07.247 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.604+0000 7f7c24a29780 10 monclient: get_monmap_and_config 2023-11-02T15:35:07.247 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.604+0000 7f7c24a29780 10 monclient: build_initial_monmap 2023-11-02T15:35:07.247 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.604+0000 7f7c24a29780 10 monclient: monmap: 2023-11-02T15:35:07.248 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 0 2023-11-02T15:35:07.248 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:35:07.248 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 0.000000 2023-11-02T15:35:07.248 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 0.000000 2023-11-02T15:35:07.248 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 0 (unknown) 2023-11-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.250 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.604+0000 7f7c24a29780 1 Processor -- start 2023-11-02T15:35:07.250 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 -- start start 2023-11-02T15:35:07.250 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient: init 2023-11-02T15:35:07.250 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient: _reopen_session rank -1 2023-11-02T15:35:07.250 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T15:35:07.251 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 0x5632d2dbb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.251 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-c con 0x5632d2d21d50 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:35:07.251 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632d2dbd230 0x5632d2db9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.252 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-a con 0x5632d2dbd230 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:35:07.252 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.252 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-b con 0x5632d2dbdc70 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:35:07.252 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.252 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d2c37280 con 0x5632d2dbd230 2023-11-02T15:35:07.253 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.253 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d2c36f20 con 0x5632d2dbdc70 2023-11-02T15:35:07.253 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.253 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d2c1a420 con 0x5632d2d21d50 2023-11-02T15:35:07.253 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): _renew_subs 2023-11-02T15:35:07.254 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.605+0000 7f7c24a29780 10 monclient(hunting): authenticate will time out at 1433.947530s 2023-11-02T15:35:07.257 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_watch_notify 2023-11-02T15:35:07.258 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.258 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.258 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_watch_notify 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91727 2023-11-02T15:35:07.263 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_watch_notify on pid 91727' 2023-11-02T15:35:07.264 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.264 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.264 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.264 INFO:tasks.workunit.client.0.smithi174.stdout: test api_watch_notify on pid 91727 2023-11-02T15:35:07.272 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.273 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91727 2023-11-02T15:35:07.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.274 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.274 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.275 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.275 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.275 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.275 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.275 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.276 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.278 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.278 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.278 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.278 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_watch_notify_pp 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91627/exe 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.280 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.281 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_watch_notify_pp' 2023-11-02T15:35:07.281 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.281 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.281 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.282 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.282 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.282 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.282 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.282 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.283 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_watch_notify_pp 2023-11-02T15:35:07.283 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.284 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.285 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.285 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.285 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.285 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.285 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.286 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.287 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.287 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.287 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.288 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.288 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.288 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2023-11-02T15:35:07.289 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.289 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.289 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.290 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_snapshots_pp 2023-11-02T15:35:07.290 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.290 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.290 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.290 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.291 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.291 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.291 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.291 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.291 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.292 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_watch_notify_pp 2023-11-02T15:35:07.292 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.292 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91741 2023-11-02T15:35:07.292 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_watch_notify_pp on pid 91741' 2023-11-02T15:35:07.293 INFO:tasks.workunit.client.0.smithi174.stdout:test api_watch_notify_pp on pid 91741 2023-11-02T15:35:07.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91741 2023-11-02T15:35:07.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2023-11-02T15:35:07.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_snapshots_pp: /' 2023-11-02T15:35:07.301 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cmd 2023-11-02T15:35:07.301 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.301 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.302 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.302 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.302 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.302 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.302 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.303 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.303 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.303 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.303 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.303 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.304 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.305 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.305 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.305 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cmd' 2023-11-02T15:35:07.305 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.305 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.306 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.306 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cmd 2023-11-02T15:35:07.306 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.306 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.306 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.307 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.308 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.308 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.308 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.308 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.308 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.309 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.316 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.316 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.316 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.316 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.316 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.317 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.317 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.317 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.317 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.317 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.318 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.319 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.319 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.319 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.330 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.330 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.331 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.331 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.331 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.331 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.331 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.332 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.332 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.332 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.332 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.333 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.333 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.333 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.333 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.334 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.334 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.334 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.334 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cmd 2023-11-02T15:35:07.335 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91756 2023-11-02T15:35:07.335 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cmd on pid 91756' 2023-11-02T15:35:07.335 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cmd on pid 91756 2023-11-02T15:35:07.341 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91756 2023-11-02T15:35:07.342 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.342 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.342 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cmd_pp 2023-11-02T15:35:07.342 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cmd_pp' 2023-11-02T15:35:07.342 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cmd_pp 2023-11-02T15:35:07.343 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.343 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2023-11-02T15:35:07.343 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.343 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.343 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.344 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.344 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.344 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.344 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.344 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.345 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.346 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.346 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.346 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.346 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.346 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.347 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.347 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.347 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.347 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.347 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.348 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.348 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.348 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.349 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.349 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.349 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.349 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.349 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.350 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.350 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.350 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.350 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.351 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.351 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.352 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.352 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.352 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.352 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.352 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.353 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.353 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.353 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.353 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.353 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.354 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.354 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.354 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.354 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.354 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cmd_pp 2023-11-02T15:35:07.355 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.355 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.355 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.355 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91765 2023-11-02T15:35:07.356 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cmd_pp on pid 91765 2023-11-02T15:35:07.364 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cmd_pp on pid 91765' 2023-11-02T15:35:07.365 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91765 2023-11-02T15:35:07.365 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.365 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.365 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2023-11-02T15:35:07.366 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_service 2023-11-02T15:35:07.366 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.366 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.366 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.366 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.367 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.368 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.368 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.368 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.368 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.368 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.369 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.369 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.369 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.369 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.369 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.370 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.370 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.370 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.370 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.371 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.372 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_service' 2023-11-02T15:35:07.372 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.372 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.372 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_service 2023-11-02T15:35:07.373 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.373 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.373 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.373 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.373 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.374 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.374 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.374 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.374 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.374 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.375 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.375 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.375 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.375 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.375 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.377 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.377 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.377 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.378 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.378 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.378 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.378 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.378 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.379 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.379 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.379 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.379 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.380 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.380 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.381 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.381 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.381 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.381 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.381 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.382 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.383 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.383 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.383 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.383 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.384 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.384 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.384 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.384 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.385 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.385 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.385 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.385 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.385 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.386 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.386 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.386 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.386 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.386 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.387 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.387 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.387 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.387 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.387 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.388 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_service 2023-11-02T15:35:07.388 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.388 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91784 2023-11-02T15:35:07.389 INFO:tasks.workunit.client.0.smithi174.stdout:test api_service on pid 91784 2023-11-02T15:35:07.405 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_service on pid 91784' 2023-11-02T15:35:07.405 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91784 2023-11-02T15:35:07.405 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.405 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_service_pp 2023-11-02T15:35:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.408 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.408 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.408 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.408 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.408 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_service_pp' 2023-11-02T15:35:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2023-11-02T15:35:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_service_pp 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.411 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.412 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.412 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.412 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.412 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.412 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.413 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.413 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.413 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.413 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.413 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.414 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.414 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.415 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.415 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.415 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.415 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.415 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.416 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.417 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.417 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.417 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.417 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.417 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.419 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.419 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.419 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.419 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.419 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.420 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.420 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.420 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.420 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.420 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.421 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.422 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.422 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.422 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.422 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.422 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.423 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.423 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.423 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.423 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.423 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.424 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.424 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.425 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.425 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.425 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.425 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.425 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.426 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.426 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.426 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.426 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.426 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.427 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.427 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.427 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.427 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.427 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.428 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.429 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91667/exe 2023-11-02T15:35:07.429 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.429 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.429 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.429 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91704/exe 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.430 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.431 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.431 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.431 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.431 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.431 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.432 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.432 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.432 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.432 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.432 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.433 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.434 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.435 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.435 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.435 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.435 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.436 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_stat 2023-11-02T15:35:07.436 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.436 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.436 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.437 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.438 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.438 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.438 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.438 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.438 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_stat.log 2023-11-02T15:35:07.439 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.439 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.439 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.439 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_stat: /' 2023-11-02T15:35:07.440 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.441 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.441 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.441 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.441 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.441 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_service_pp 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91817 2023-11-02T15:35:07.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_service_pp on pid 91817' 2023-11-02T15:35:07.443 INFO:tasks.workunit.client.0.smithi174.stdout:test api_service_pp on pid 91817 2023-11-02T15:35:07.454 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91817 2023-11-02T15:35:07.455 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.456 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.456 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.456 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.456 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.457 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.458 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.458 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.458 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.459 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.459 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.459 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.460 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.460 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.460 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.460 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.461 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.461 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_stat_pp.log 2023-11-02T15:35:07.461 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.461 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.462 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.463 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.463 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.463 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.463 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_stat_pp: /' 2023-11-02T15:35:07.463 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.464 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.464 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.464 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.464 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.464 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.465 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.465 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.465 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.465 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.465 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.466 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_c_write_operations 2023-11-02T15:35:07.466 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.467 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.467 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.467 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.467 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.467 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_stat_pp 2023-11-02T15:35:07.468 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.468 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.468 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.468 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.468 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_c_write_operations' 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.469 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.470 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.470 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.470 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.470 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.470 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.471 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.471 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.471 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.471 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.471 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.472 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.472 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.472 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.472 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_c_write_operations 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.473 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.474 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.474 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.474 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.474 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.474 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.475 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.475 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.475 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.475 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.475 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.476 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.476 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.476 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.477 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.478 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.478 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.478 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.478 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.479 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.479 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.480 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.480 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.480 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.480 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.481 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.483 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_c_write_operations 2023-11-02T15:35:07.483 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91844 2023-11-02T15:35:07.484 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.484 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.485 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.485 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_c_write_operations on pid 91844' 2023-11-02T15:35:07.485 INFO:tasks.workunit.client.0.smithi174.stdout:test api_c_write_operations on pid 91844 2023-11-02T15:35:07.499 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91844 2023-11-02T15:35:07.499 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.500 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.500 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.500 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.500 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.500 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.501 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.502 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.502 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.502 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.502 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.502 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91727/exe 2023-11-02T15:35:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_c_read_operations 2023-11-02T15:35:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_c_read_operations' 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.508 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.508 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.508 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.508 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.508 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.510 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.510 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.510 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.510 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.510 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.511 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.512 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.512 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.512 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.512 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.512 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_watch_notify 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.513 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.514 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.514 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.515 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.515 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.515 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.515 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.515 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.516 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.516 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.517 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.517 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.518 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.518 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.518 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.519 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.519 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.519 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.519 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.519 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.520 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.521 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.522 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.523 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.523 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.523 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.523 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.523 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.524 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.525 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.525 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.525 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.525 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.525 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.526 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.526 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.526 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.526 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.526 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91741/exe 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.527 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.528 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.528 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.529 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91756/exe 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_watch_notify.log 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.530 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.531 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.531 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.531 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.531 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.531 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.532 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.532 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_watch_notify_pp 2023-11-02T15:35:07.532 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.532 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.532 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.533 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.534 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.534 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.534 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.534 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.534 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.535 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.536 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.609+0000 7f7c21e1c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:35:07.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:56664/0 (socket says 172.21.15.174:56664) 2023-11-02T15:35:07.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 1 -- 172.21.15.174:0/2142992182 learned_addr learned my addr 172.21.15.174:0/2142992182 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:35:07.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 10 monclient(hunting): get_auth_request con 0x5632d2dbdc70 auth_method 0 2023-11-02T15:35:07.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:35:07.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:35:07.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.617+0000 7f7c21e1c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:35:07.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.619+0000 7f7c21e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:35:07.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.619+0000 7f7c21e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:35:07.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.619+0000 7f7c21e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:35:07.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 10 monclient(hunting): handle_auth_done global_id 4429 payload 274 2023-11-02T15:35:07.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 -- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 msgr2=0x5632d2dbb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:35:07.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 0x5632d2dbb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 -- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632d2dbd230 msgr2=0x5632d2db9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T15:35:07.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632d2dbd230 0x5632d2db9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 10 monclient: _finish_hunting 0 2023-11-02T15:35:07.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 monclient: found mon.noname-b 2023-11-02T15:35:07.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:35:07.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:35:07.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.634+0000 7f7c21e1c640 1 -- 172.21.15.174:0/2142992182 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18005de0 con 0x5632d2dbdc70 2023-11-02T15:35:07.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.638+0000 7f7c21e1c640 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00a4b0 tx=0x7f7c1c00a970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ade44c6e1741083 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:35:07.542 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.542 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.542 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.542 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.543 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.543 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.543 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.543 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.543 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 1 -- 172.21.15.174:0/2142992182 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00f7b0 con 0x5632d2dbdc70 2023-11-02T15:35:07.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:35:07.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:35:07.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: dump: 2023-11-02T15:35:07.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 1 2023-11-02T15:35:07.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:07.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 18 (reef) 2023-11-02T15:35:07.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:35:07.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:35:07.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:35:07.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: _finish_auth 0 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: _check_auth_tickets 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 1 -- 172.21.15.174:0/2142992182 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00f910 con 0x5632d2dbdc70 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:35:07.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 1 -- 172.21.15.174:0/2142992182 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00fca0 con 0x5632d2dbdc70 2023-11-02T15:35:07.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:35:07.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:35:07.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c137fe640 10 monclient: dump: 2023-11-02T15:35:07.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 1 2023-11-02T15:35:07.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:07.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 18 (reef) 2023-11-02T15:35:07.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:35:07.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c24a29780 5 monclient: authenticate success, global_id 4429 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c24a29780 10 monclient: get_monmap_and_config success 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.678+0000 7f7c24a29780 4 set_mon_vals no callback set 2023-11-02T15:35:07.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.679+0000 7f7c24a29780 10 monclient: shutdown 2023-11-02T15:35:07.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.679+0000 7f7c24a29780 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:35:07.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.679+0000 7f7c24a29780 1 -- 172.21.15.174:0/2142992182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 msgr2=0x5632d2dbc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:07.555 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.555 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.556 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.557 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.557 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.558 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.571 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.571 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.571 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.572 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_watch_notify: /' 2023-11-02T15:35:07.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.572 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.573 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.573 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.573 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.574 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.574 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.574 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.574 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.574 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2023-11-02T15:35:07.575 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_c_read_operations 2023-11-02T15:35:07.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.679+0000 7f7c24a29780 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00a4b0 tx=0x7f7c1c00a970 comp rx=0 tx=0).stop 2023-11-02T15:35:07.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/2142992182 shutdown_connections 2023-11-02T15:35:07.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbdc70 0x5632d2dbc2a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632d2dbd230 0x5632d2db9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/2142992182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 0x5632d2dbb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/2142992182 >> 172.21.15.174:0/2142992182 conn(0x5632d2ce2870 msgr2=0x5632d2d20700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:35:07.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/2142992182 shutdown_connections 2023-11-02T15:35:07.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/2142992182 wait complete. 2023-11-02T15:35:07.577 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 10 monclient: build_initial_monmap 2023-11-02T15:35:07.577 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 10 monclient: monmap: 2023-11-02T15:35:07.577 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 0 2023-11-02T15:35:07.577 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2023-11-02T15:35:07.577 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 0.000000 2023-11-02T15:35:07.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 0.000000 2023-11-02T15:35:07.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 0 (unknown) 2023-11-02T15:35:07.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T15:35:07.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T15:35:07.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T15:35:07.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 librados: starting msgr at 2023-11-02T15:35:07.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 librados: starting objecter 2023-11-02T15:35:07.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 Processor -- start 2023-11-02T15:35:07.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 -- start start 2023-11-02T15:35:07.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 librados: setting wanted keys 2023-11-02T15:35:07.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 1 librados: calling monclient init 2023-11-02T15:35:07.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.681+0000 7f7c24a29780 10 monclient: init 2023-11-02T15:35:07.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient: _reopen_session rank -1 2023-11-02T15:35:07.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T15:35:07.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 0x5632d2da49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-c con 0x5632d2d21d50 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T15:35:07.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 0x5632d2db8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-b con 0x5632d2dbd230 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T15:35:07.590 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_c_read_operations 2023-11-02T15:35:07.590 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91908 2023-11-02T15:35:07.590 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_c_read_operations on pid 91908' 2023-11-02T15:35:07.591 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 --2- >> [v2:172.21.15.174:3300/0,v1:test api_c_read_operations on pid 91908 2023-11-02T15:35:07.595 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91908 2023-11-02T15:35:07.595 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.596 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.596 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.596 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cls_remote_reads 2023-11-02T15:35:07.596 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.596 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.597 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.597 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cls_remote_reads' 2023-11-02T15:35:07.597 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.598 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.598 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cmd 2023-11-02T15:35:07.598 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.598 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.598 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.599 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.599 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cmd.log 2023-11-02T15:35:07.599 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cmd: /' 2023-11-02T15:35:07.599 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.599 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.607 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.608 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.608 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.608 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.608 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.608 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.609 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cls_remote_reads 2023-11-02T15:35:07.609 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.609 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.609 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cls_remote_reads 2023-11-02T15:35:07.609 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.610 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.610 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.610 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91918 2023-11-02T15:35:07.610 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cls_remote_reads on pid 91918' 2023-11-02T15:35:07.611 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cls_remote_reads on pid 91918 2023-11-02T15:35:07.669 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91918 2023-11-02T15:35:07.669 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.669 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.670 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.670 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.670 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.670 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.670 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.672 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.672 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.672 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.672 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.673 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.673 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.673 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.674 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.674 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.674 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.674 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.674 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.675 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.675 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.675 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.675 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.675 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2023-11-02T15:35:07.676 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2023-11-02T15:35:07.676 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.676 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.676 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.676 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.677 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.677 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.677 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.679 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.679 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.680 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.680 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.680 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.680 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.680 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.681 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.681 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.681 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.681 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.681 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.682 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.682 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.682 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.683 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.684 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.690 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.690 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.690 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.690 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.691 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.691 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.691 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.691 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.691 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.692 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.693 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.693 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.693 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.693 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.693 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.694 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.695 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.696 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.697 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.697 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.697 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.697 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.697 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.698 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.698 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.699 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.700 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.700 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.700 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.701 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.701 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.701 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.701 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.701 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.702 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.702 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.702 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.702 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.702 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.703 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.703 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91765/exe 2023-11-02T15:35:07.703 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.703 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.703 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.704 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.704 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.704 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.704 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.704 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.705 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.706 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.706 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.706 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cmd_pp 2023-11-02T15:35:07.706 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s list_parallel 2023-11-02T15:35:07.707 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2023-11-02T15:35:07.707 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' list_parallel' 2023-11-02T15:35:07.707 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2023-11-02T15:35:07.707 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo list_parallel 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.708 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.709 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.709 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.709 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.709 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.709 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.710 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.710 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.710 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.710 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.710 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.711 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.711 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.711 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.711 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.711 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.712 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.713 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.713 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.713 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.713 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.713 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.714 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.714 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.714 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.714 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.714 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.715 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.715 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.716 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.716 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.716 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.716 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.717 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.717 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.718 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.718 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.718 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.718 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.718 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.719 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.720 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.720 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.720 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.720 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.720 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.721 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.722 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.722 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91817/exe 2023-11-02T15:35:07.722 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.722 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=list_parallel 2023-11-02T15:35:07.722 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91959 2023-11-02T15:35:07.723 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test list_parallel on pid 91959' 2023-11-02T15:35:07.723 INFO:tasks.workunit.client.0.smithi174.stdout:test list_parallel on pid 91959 2023-11-02T15:35:07.725 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=91959 2023-11-02T15:35:07.725 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.725 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.725 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.727 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.727 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.727 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.727 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.727 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.728 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.728 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.728 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.728 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.728 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.729 INFO:tasks.workunit.client.0.smithi174.stdout:172.21.15.174:6789/0] conn(0x5632d2dbdc70 0x5632d2db8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): picked mon.noname-a con 0x5632d2dbdc70 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T15:35:07.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d2da4530 con 0x5632d2dbdc70 2023-11-02T15:35:07.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d2da86f0 con 0x5632d2dbd230 2023-11-02T15:35:07.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): start opening mon connection 2023-11-02T15:35:07.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d2da89d0 con 0x5632d2d21d50 2023-11-02T15:35:07.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): _renew_subs 2023-11-02T15:35:07.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.682+0000 7f7c24a29780 10 monclient(hunting): authenticate will time out at 1434.024277s 2023-11-02T15:35:07.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.707+0000 7f7c20e1a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 0x5632d2db8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:35:07.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 0x5632d2db8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:56734/0 (socket says 172.21.15.174:56734) 2023-11-02T15:35:07.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 1 -- 172.21.15.174:0/1305488291 learned_addr learned my addr 172.21.15.174:0/1305488291 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T15:35:07.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 10 monclient(hunting): get_auth_request con 0x5632d2dbd230 auth_method 0 2023-11-02T15:35:07.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T15:35:07.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 10 monclient(hunting): _init_auth method 2 2023-11-02T15:35:07.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.708+0000 7f7c20e1a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T15:35:07.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.710+0000 7f7c20e1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T15:35:07.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.710+0000 7f7c20e1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T15:35:07.733 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.710+0000 7f7c20e1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T15:35:07.733 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2023-11-02T15:35:07.733 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 -- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 msgr2=0x5632d2da49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:07.733 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5632d2d21d50 0x5632d2da49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.758 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cmd_pp: /' 2023-11-02T15:35:07.758 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91784/exe 2023-11-02T15:35:07.759 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_service_pp: /' 2023-11-02T15:35:07.759 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.759 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.759 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.760 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.760 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.761 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.761 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.761 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.761 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.761 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.762 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.762 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.763 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.763 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.763 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.763 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.764 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.764 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.764 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.764 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.764 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.765 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.765 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.765 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.765 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.765 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_service_pp.log 2023-11-02T15:35:07.766 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_service_pp 2023-11-02T15:35:07.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_service: /' 2023-11-02T15:35:07.766 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.767 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.767 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.767 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.767 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.768 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.768 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.768 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.769 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.769 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.770 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.770 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.770 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.771 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.773 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.773 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2023-11-02T15:35:07.773 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.773 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.774 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.774 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.774 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.775 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s open_pools_parallel 2023-11-02T15:35:07.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.775 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' open_pools_parallel' 2023-11-02T15:35:07.776 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.776 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.776 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.780 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_service.log 2023-11-02T15:35:07.780 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.781 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.781 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.782 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.782 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.783 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.792 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_service 2023-11-02T15:35:07.793 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.793 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.793 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.793 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.794 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.794 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.794 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.794 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.794 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.795 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.795 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.795 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.795 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.795 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.796 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.797 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.799 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.802 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.803 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.804 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.804 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.804 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.804 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.805 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.805 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.805 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.805 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.806 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.806 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.806 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.807 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.807 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.807 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.807 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.807 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.809 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.809 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.811 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.812 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo open_pools_parallel 2023-11-02T15:35:07.814 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.816 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.816 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.816 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.816 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.816 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.817 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.817 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.817 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.817 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.817 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.818 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.818 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.818 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.818 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.819 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.819 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.819 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.819 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.819 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.820 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.820 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.820 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91908/exe 2023-11-02T15:35:07.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.821 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.821 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.822 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.824 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.824 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.828 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.828 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.828 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=open_pools_parallel 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91844/exe 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=92044 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2023-11-02T15:35:07.829 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test open_pools_parallel on pid 92044' 2023-11-02T15:35:07.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 -- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:3300/0,v1:172.21.15test open_pools_parallel on pid 92044 2023-11-02T15:35:07.846 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=92044 2023-11-02T15:35:07.846 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-11-02T15:35:07.846 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.846 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2023-11-02T15:35:07.847 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.848 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.848 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.848 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_c_read_operations 2023-11-02T15:35:07.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.849 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.849 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.849 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.849 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.849 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.850 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.850 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.850 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.850 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_c_read_operations: /' 2023-11-02T15:35:07.850 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.851 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.851 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.851 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.851 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.851 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.852 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.852 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.852 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.855 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.855 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.855 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.855 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s delete_pools_parallel 2023-11-02T15:35:07.861 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' delete_pools_parallel' 2023-11-02T15:35:07.861 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91918/exe 2023-11-02T15:35:07.861 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.862 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.866 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.866 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.866 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.867 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.867 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.867 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.867 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.868 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.869 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.869 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cls_remote_reads 2023-11-02T15:35:07.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2023-11-02T15:35:07.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.872 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2023-11-02T15:35:07.873 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.873 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_c_write_operations 2023-11-02T15:35:07.873 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_c_write_operations: /' 2023-11-02T15:35:07.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2023-11-02T15:35:07.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.874 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.874 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.875 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.876 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.876 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.876 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.877 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.877 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.877 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.877 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.877 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.878 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.878 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.878 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.878 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.878 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.879 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.879 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.879 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.879 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/91959/exe 2023-11-02T15:35:07.879 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.880 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.881 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.881 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.881 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.881 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo delete_pools_parallel 2023-11-02T15:35:07.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2023-11-02T15:35:07.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.885 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.885 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.887 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=delete_pools_parallel 2023-11-02T15:35:07.888 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_list_parallel 2023-11-02T15:35:07.888 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=92084 2023-11-02T15:35:07.888 INFO:tasks.workunit.client.0.smithi174.stdout:test delete_pools_parallel on pid 92084 2023-11-02T15:35:07.899 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test delete_pools_parallel on pid 92084' 2023-11-02T15:35:07.899 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=92084 2023-11-02T15:35:07.900 INFO:tasks.workunit.client.0.smithi174.stderr:+ ret=0 2023-11-02T15:35:07.900 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2023-11-02T15:35:07.900 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T15:35:07.901 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91458 2023-11-02T15:35:07.901 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91458 2023-11-02T15:35:07.901 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2023-11-02T15:35:07.901 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2023-11-02T15:35:07.901 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2023-11-02T15:35:07.902 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2023-11-02T15:35:07.902 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2023-11-02T15:35:07.902 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.902 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2023-11-02T15:35:07.903 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2023-11-02T15:35:07.903 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2023-11-02T15:35:07.903 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2023-11-02T15:35:07.903 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.903 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-11-02T15:35:07.904 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.905 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ list_parallel: /' 2023-11-02T15:35:07.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2023-11-02T15:35:07.906 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.906 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-11-02T15:35:07.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-11-02T15:35:07.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2023-11-02T15:35:07.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.916 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.917 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.917 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.917 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.918 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.918 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.919 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.919 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.919 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.919 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.919 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_list_parallel.log 2023-11-02T15:35:07.920 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.920 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.920 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.922 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.922 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.922 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.922 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.922 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/92044/exe 2023-11-02T15:35:07.935 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.936 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.944 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.944 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.945 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.945 INFO:tasks.workunit.client.0.smithi174.stdout:.174:6789/0] conn(0x5632d2dbdc70 msgr2=0x5632d2db8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:07.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632d2dbdc70 0x5632d2db8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:07.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 10 monclient: _finish_hunting 0 2023-11-02T15:35:07.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 monclient: found mon.noname-b 2023-11-02T15:35:07.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:35:07.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T15:35:07.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.713+0000 7f7c20e1a640 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c00a0e0 con 0x5632d2dbd230 2023-11-02T15:35:07.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.757+0000 7f7c20e1a640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 0x5632d2db8800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801cdb0 tx=0x7f7c1804ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:35:07.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.757+0000 7f7c127fc640 10 client.?.objecter ms_handle_connect 0x5632d2dbd230 2023-11-02T15:35:07.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.757+0000 7f7c127fc640 10 client.?.objecter resend_mon_ops 2023-11-02T15:35:07.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180529c0 con 0x5632d2dbd230 2023-11-02T15:35:07.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:35:07.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T15:35:07.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: dump: 2023-11-02T15:35:07.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 1 2023-11-02T15:35:07.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:07.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 18 (reef) 2023-11-02T15:35:07.950 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.950 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:35:07.950 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:35:07.950 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:35:07.950 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: _finish_auth 0 2023-11-02T15:35:07.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: _check_auth_tickets 2023-11-02T15:35:07.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:07.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1804ae90 con 0x5632d2dbd230 2023-11-02T15:35:07.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T15:35:07.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18052d60 con 0x5632d2dbd230 2023-11-02T15:35:07.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T15:35:07.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T15:35:07.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_open_pools_parallel 2023-11-02T15:35:07.963 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ open_pools_parallel: /' 2023-11-02T15:35:07.963 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2023-11-02T15:35:07.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2023-11-02T15:35:07.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2023-11-02T15:35:07.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-11-02T15:35:07.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-11-02T15:35:07.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2023-11-02T15:35:07.965 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2023-11-02T15:35:07.965 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2023-11-02T15:35:07.965 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.966 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-11-02T15:35:07.966 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.966 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-11-02T15:35:07.966 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2023-11-02T15:35:07.966 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2023-11-02T15:35:07.967 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.967 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-11-02T15:35:07.967 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.967 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-11-02T15:35:07.968 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.968 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-11-02T15:35:07.969 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-11-02T15:35:07.969 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-11-02T15:35:07.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-11-02T15:35:07.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-11-02T15:35:07.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-11-02T15:35:07.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2023-11-02T15:35:07.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-11-02T15:35:07.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-11-02T15:35:07.971 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-11-02T15:35:07.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-11-02T15:35:07.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2023-11-02T15:35:07.972 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-11-02T15:35:07.972 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.972 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-11-02T15:35:07.972 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.972 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2023-11-02T15:35:07.973 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.973 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-11-02T15:35:07.973 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.973 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2023-11-02T15:35:07.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2023-11-02T15:35:07.974 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2023-11-02T15:35:07.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2023-11-02T15:35:07.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.975 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2023-11-02T15:35:07.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2023-11-02T15:35:07.976 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-11-02T15:35:07.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-11-02T15:35:07.976 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2023-11-02T15:35:07.976 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2023-11-02T15:35:07.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-11-02T15:35:07.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-11-02T15:35:07.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2023-11-02T15:35:07.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2023-11-02T15:35:07.978 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2023-11-02T15:35:07.978 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-11-02T15:35:07.978 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2023-11-02T15:35:07.980 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.980 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-11-02T15:35:07.980 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.980 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2023-11-02T15:35:07.980 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-11-02T15:35:07.981 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2023-11-02T15:35:07.981 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-11-02T15:35:07.981 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2023-11-02T15:35:07.982 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2023-11-02T15:35:07.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-11-02T15:35:07.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/92084/exe 2023-11-02T15:35:07.982 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2023-11-02T15:35:07.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2023-11-02T15:35:07.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2023-11-02T15:35:07.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2023-11-02T15:35:07.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2023-11-02T15:35:07.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2023-11-02T15:35:07.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2023-11-02T15:35:07.984 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-11-02T15:35:07.984 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2023-11-02T15:35:07.984 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-11-02T15:35:07.985 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2023-11-02T15:35:07.985 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_delete_pools_parallel 2023-11-02T15:35:07.985 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2023-11-02T15:35:07.986 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ delete_pools_parallel: /' 2023-11-02T15:35:07.990 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c127fc640 10 monclient: dump: 2023-11-02T15:35:07.990 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: epoch 1 2023-11-02T15:35:07.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:07.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: created 2023-11-02T15:32:14.751315+0000 2023-11-02T15:35:07.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: min_mon_release 18 (reef) 2023-11-02T15:35:07.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: election_strategy: 1 2023-11-02T15:35:07.992 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T15:35:07.992 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T15:35:07.992 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T15:35:07.992 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.992 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 5 monclient: authenticate success, global_id 4450 2023-11-02T15:35:07.993 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 monclient: _renew_subs 2023-11-02T15:35:07.993 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:35:07.993 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632d2c46340 con 0x5632d2dbd230 2023-11-02T15:35:07.993 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:35:07.993 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 monclient: _renew_subs 2023-11-02T15:35:07.994 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:35:07.994 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d2c39780 con 0x5632d2dbd230 2023-11-02T15:35:07.994 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 1 librados: init done 2023-11-02T15:35:07.994 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c24a29780 10 librados: wait_for_osdmap waiting 2023-11-02T15:35:07.994 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.802+0000 7f7c137fe640 4 set_mon_vals no callback set 2023-11-02T15:35:07.995 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.855+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c18059070 con 0x5632d2dbd230 2023-11-02T15:35:07.995 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.855+0000 7f7c127fc640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f7c0406a030 0x7f7c0406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:07.995 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.856+0000 7f7c2161b640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f7c0406a030 0x7f7c0406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:35:07.995 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.856+0000 7f7c2161b640 10 monclient: get_auth_request con 0x7f7c0406a030 auth_method 0 2023-11-02T15:35:07.995 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.856+0000 7f7c2161b640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f7c0406a030 0x7f7c0406c4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7c14001d70 tx=0x7f7c14001370 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:35:07.996 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.857+0000 7f7c127fc640 10 client.4450.objecter ms_handle_connect 0x7f7c0406a030 2023-11-02T15:35:07.996 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7c18098380 con 0x5632d2dbd230 2023-11-02T15:35:07.996 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(22..22 src has 1..22) v4 2023-11-02T15:35:07.997 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [22,22] > 0 2023-11-02T15:35:07.997 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding full epoch 22 2023-11-02T15:35:07.997 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:07.997 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c24a29780 10 librados: wait_for_osdmap done waiting 2023-11-02T15:35:07.997 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c24a29780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:35:07.999 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.858+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c18052440 con 0x5632d2dbd230 2023-11-02T15:35:07.999 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.890+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c18093fc0 con 0x5632d2dbd230 2023-11-02T15:35:07.999 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.890+0000 7f7c127fc640 10 monclient: handle_get_version_reply finishing 1 version 22 2023-11-02T15:35:07.999 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.890+0000 7f7c12ffd640 10 client.4450.objecter _get_latest_version latest 22, have it 2023-11-02T15:35:07.999 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.891+0000 7f7c24a29780 10 client.4450.objecter create_pool name=ceph_test_rados_api_asio 2023-11-02T15:35:08.000 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.891+0000 7f7c24a29780 10 client.4450.objecter pool_op_submit 1 2023-11-02T15:35:08.000 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.891+0000 7f7c24a29780 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:35:08.001 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:06.891+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x5632d2ca5780 con 0x5632d2dbd230 2023-11-02T15:35:08.001 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v23) v1 ==== 43+0+0 (secure 0 0 0) 0x7f7c18057050 con 0x5632d2dbd230 2023-11-02T15:35:08.001 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 pool_op_reply(tid 1 (0) Success v23) v1 2023-11-02T15:35:08.001 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 client.4450.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v23) v1 2023-11-02T15:35:08.001 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 client.4450.objecter have request 1 at 0x5632d2da8080 Op: create 2023-11-02T15:35:08.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 20 client.4450.objecter waiting for client to reach epoch 23 before calling back 2023-11-02T15:35:08.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T15:35:08.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 monclient: _renew_subs 2023-11-02T15:35:08.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T15:35:08.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=23}) v3 -- 0x7f7c0406fd70 con 0x5632d2dbd230 2023-11-02T15:35:08.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.865+0000 7f7c127fc640 10 client.4450.objecter done 2023-11-02T15:35:08.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.880+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== mon.1 v2:172.21.15.175:3300/0 8 ==== osd_map(23..23 src has 1..23) v4 ==== 3603+0+0 (secure 0 0 0) 0x7f7c18097c80 con 0x5632d2dbd230 2023-11-02T15:35:08.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.880+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(23..23 src has 1..23) v4 2023-11-02T15:35:08.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.880+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [23,23] > 22 2023-11-02T15:35:10.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.880+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 23 2023-11-02T15:35:10.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:10.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 10 librados: set snap read head -> 1 2023-11-02T15:35:10.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 10 librados: writefull oid=exist nspace= 2023-11-02T15:35:10.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:10.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 23 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:10.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:10.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:10.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x5632d2db9280 0x5632d2e5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T15:35:10.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:10.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [writefull 0~5 in=5b] tid 2 osd.0 2023-11-02T15:35:10.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:10.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 2 2023-11-02T15:35:10.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 15 client.4450.objecter _send_op 2 to 10.1 on osd.0 2023-11-02T15:35:10.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:2 10.1 10:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23) v8 -- 0x5632d2e5fb10 con 0x5632d2db9280 2023-11-02T15:35:10.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:10.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:10.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c21e1c640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x5632d2db9280 0x5632d2e5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T15:35:10.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.881+0000 7f7c21e1c640 10 monclient: get_auth_request con 0x5632d2db9280 auth_method 0 2023-11-02T15:35:10.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.884+0000 7f7c127fc640 10 client.4450.objecter ms_handle_connect 0x5632d2db9280 2023-11-02T15:35:10.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:07.884+0000 7f7c21e1c640 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x5632d2db9280 0x5632d2e5f5d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T15:35:10.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:09.682+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:10.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:09.682+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:10.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:09.682+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:10.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:09.682+0000 7f7c117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:35:10.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:10.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:10.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:10.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:18.339 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2023-11-02T15:35:18.339 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] Global test environment set-up. 2023-11-02T15:35:18.339 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] 4 tests from LibRadosService 2023-11-02T15:35:18.340 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2023-11-02T15:35:18.340 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5357 ms) 2023-11-02T15:35:18.340 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2023-11-02T15:35:18.340 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.RegisterLate (63 ms) 2023-11-02T15:35:18.340 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2023-11-02T15:35:18.341 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2023-11-02T15:35:18.341 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:18.341 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2023-11-02T15:35:18.341 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 5 pool(s) do not have an application enabled 2023-11-02T15:35:18.341 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.342 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2023-11-02T15:35:18.342 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-02T15:35:18.342 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 2m), standbys: x 2023-11-02T15:35:18.342 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 80s), 8 in (since 2m) 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: laundry: 2 daemons active (1 hosts) 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 26 pools, 777 pgs 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 2 objects, 449 KiB 2023-11-02T15:35:18.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 213 MiB used, 720 GiB / 720 GiB avail 2023-11-02T15:35:18.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 88.932% pgs unknown 2023-11-02T15:35:18.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 9.910% pgs not active 2023-11-02T15:35:18.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 691 unknown 2023-11-02T15:35:18.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 77 creating+peering 2023-11-02T15:35:18.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 9 active+clean 2023-11-02T15:35:18.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2023-11-02T15:35:18.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:18.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2023-11-02T15:35:18.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 8 pool(s) do not have an application enabled 2023-11-02T15:35:18.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2023-11-02T15:35:18.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-02T15:35:18.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 2m), standbys: x 2023-11-02T15:35:18.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 82s), 8 in (since 2m) 2023-11-02T15:35:18.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: foo: 6 portals active (1 hosts, 3 zones) 2023-11-02T15:35:18.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.348 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2023-11-02T15:35:18.348 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 27 pools, 785 pgs 2023-11-02T15:35:18.348 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 2 objects, 449 KiB 2023-11-02T15:35:18.348 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 213 MiB used, 720 GiB / 720 GiB avail 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 71.338% pgs unknown 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 26.497% pgs not active 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 560 unknown 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 186 creating+peering 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 22 creating+activating 2023-11-02T15:35:18.349 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 17 active+clean 2023-11-02T15:35:18.350 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.350 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.350 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2023-11-02T15:35:18.350 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T15:35:18.350 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2023-11-02T15:35:18.351 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 7 pool(s) do not have an application enabled 2023-11-02T15:35:18.351 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.351 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2023-11-02T15:35:18.351 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-02T15:35:18.351 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 2m), standbys: x 2023-11-02T15:35:18.352 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 85s), 8 in (since 2m) 2023-11-02T15:35:18.352 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2023-11-02T15:35:18.352 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: laundry: 1 daemon active (1 hosts) 2023-11-02T15:35:18.352 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:18.352 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2023-11-02T15:35:18.353 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 29 pools, 849 pgs 2023-11-02T15:35:18.353 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 3 objects, 449 KiB 2023-11-02T15:35:18.353 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 214 MiB used, 720 GiB / 720 GiB avail 2023-11-02T15:35:18.353 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 35.100% pgs unknown 2023-11-02T15:35:19.431 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:10.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.431 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:11.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.432 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:11.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.432 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:11.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.432 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:11.803+0000 7f7c2387f640 10 client.4450.objecter tick 2023-11-02T15:35:19.432 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:12.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.432 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:12.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.433 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:12.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.433 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:13.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.433 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:13.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.433 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:13.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.433 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:14.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.434 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:14.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.434 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:14.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.434 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:15.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.434 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:15.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:15.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:16.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:16.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:16.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:16.804+0000 7f7c2387f640 10 client.4450.objecter tick 2023-11-02T15:35:19.435 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:17.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:17.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:17.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:18.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:19.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:18.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:19.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:18.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:19.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 1 ==== osd_map(24..26 src has 1..26) v4 ==== 16196+0+0 (crc 0 0 0) 0x7f7c1c005810 con 0x5632d2db9280 2023-11-02T15:35:19.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 2 ==== osd_op_reply(2 exist [writefull 0~5] v26'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:19.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(24..26 src has 1..26) v4 2023-11-02T15:35:19.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [24,26] > 23 2023-11-02T15:35:19.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 24 2023-11-02T15:35:19.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(2 exist [writefull 0~5] v26'1 uv1 ondisk = 0) v8 2023-11-02T15:35:19.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:19.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 10 client.4450.objecter checking op 2 2023-11-02T15:35:19.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 24 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:19.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:19.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 25 2023-11-02T15:35:19.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 10 client.4450.objecter checking op 2 2023-11-02T15:35:19.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2023-11-02T15:35:19.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 25 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:19.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 26 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 10 client.4450.objecter checking op 2 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 26 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:19.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c127fc640 20 client.4450.objecter 2 10.d3a59201 osd.0 exist [writefull 0~5 in=5b] 2023-11-02T15:35:19.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 2 ondisk uv 1 in 10.1 attempt 0 2023-11-02T15:35:19.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:19.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 2 2023-11-02T15:35:19.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 2 2023-11-02T15:35:19.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:19.442 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 2 2023-11-02T15:35:19.442 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:19.443 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 librados: Objecter returned from writefull r=0 2023-11-02T15:35:19.443 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:19.443 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 26 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:19.443 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:19.443 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:19.444 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:19.444 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~256] tid 3 osd.0 2023-11-02T15:35:19.444 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:19.444 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 3 2023-11-02T15:35:19.444 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 15 client.4450.objecter _send_op 3 to 10.1 on osd.0 2023-11-02T15:35:23.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:3 10.1 10:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) v8 -- 0x5632d2da6600 con 0x5632d2db9280 2023-11-02T15:35:23.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:23.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:23.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da69b0 2023-11-02T15:35:23.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 26 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:23.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:23.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:23.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:23.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~256] tid 4 osd.0 2023-11-02T15:35:23.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:23.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 4 2023-11-02T15:35:23.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 15 client.4450.objecter _send_op 4 to 10.1 on osd.0 2023-11-02T15:35:23.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:4 10.1 10:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) v8 -- 0x5632d2da6ea0 con 0x5632d2db9280 2023-11-02T15:35:23.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:23.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.429+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:23.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:23.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:23.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:23.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:19.683+0000 7f7c117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:35:23.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:20.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:23.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:20.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:23.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:20.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:23.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:21.682+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:23.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:21.682+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:23.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:21.682+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:23.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:21.804+0000 7f7c2387f640 10 client.4450.objecter tick 2023-11-02T15:35:23.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:22.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:23.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:22.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:23.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:22.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:23.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:23.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:23.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:23.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:24.255 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:23.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:24.255 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 3 ==== osd_map(27..33 src has 1..33) v4 ==== 5801+0+0 (crc 0 0 0) 0x7f7c1c021040 con 0x5632d2db9280 2023-11-02T15:35:24.255 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(27..33 src has 1..33) v4 2023-11-02T15:35:24.256 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 4 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:24.256 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-11-02T15:35:24.256 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:24.256 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [27,33] > 26 2023-11-02T15:35:24.256 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 27 2023-11-02T15:35:24.257 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.257 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 27 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.257 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.257 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (4824 ms) 2023-11-02T15:35:24.258 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2023-11-02T15:35:24.258 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.258 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 27 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.258 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.258 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 28 2023-11-02T15:35:24.259 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.259 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 28 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.259 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.259 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.260 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 28 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.260 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.260 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 29 2023-11-02T15:35:24.260 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.260 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 29 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.261 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.261 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.261 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 29 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.261 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.261 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 30 2023-11-02T15:35:24.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 30 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.251+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 30 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 31 2023-11-02T15:35:24.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 31 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 31 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 32 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 32 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 32 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 33 2023-11-02T15:35:24.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 3 2023-11-02T15:35:24.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 33 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 10 client.4450.objecter checking op 4 2023-11-02T15:35:24.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 33 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:24.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:24.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter 3 10.d3a59201 osd.0 exist [read 0~256] 2023-11-02T15:35:24.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c127fc640 20 client.4450.objecter 4 10.a7bbaa1 osd.0 noexist [read 0~256] 2023-11-02T15:35:24.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 3 ondisk uv 1 in 10.1 attempt 0 2023-11-02T15:35:24.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:24.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 3 2023-11-02T15:35:24.272 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 3 2023-11-02T15:35:24.272 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:24.272 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 3 2023-11-02T15:35:24.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:24.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 5 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:24.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:24.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:24.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 4 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:24.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:24.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:24.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 4 2023-11-02T15:35:24.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 4 2023-11-02T15:35:24.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:24.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 4 2023-11-02T15:35:24.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:24.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da69b0 2023-11-02T15:35:24.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 33 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:24.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:24.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:24.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:24.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~256] tid 5 osd.0 2023-11-02T15:35:24.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:24.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 5 2023-11-02T15:35:24.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 15 client.4450.objecter _send_op 5 to 10.1 on osd.0 2023-11-02T15:35:25.009 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:5 10.1 10:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5632d2da7250 con 0x5632d2db9280 2023-11-02T15:35:25.010 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:25.010 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:25.010 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:25.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 33 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:25.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:25.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:25.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:25.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~256] tid 6 osd.0 2023-11-02T15:35:25.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:25.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 6 2023-11-02T15:35:25.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 15 client.4450.objecter _send_op 6 to 10.1 on osd.0 2023-11-02T15:35:25.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:6 10.1 10:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5632d2ceee80 con 0x5632d2db9280 2023-11-02T15:35:25.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:25.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.252+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:25.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:25.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:25.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:24.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:25.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 6 ==== osd_map(34..34 src has 1..34) v4 ==== 644+0+0 (crc 0 0 0) 0x7f7c1c00fdc0 con 0x5632d2db9280 2023-11-02T15:35:25.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 7 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:25.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(34..34 src has 1..34) v4 2023-11-02T15:35:25.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-11-02T15:35:25.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:25.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [34,34] > 33 2023-11-02T15:35:25.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 34 2023-11-02T15:35:25.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 10 client.4450.objecter checking op 5 2023-11-02T15:35:25.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 34 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:25.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:25.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 10 client.4450.objecter checking op 6 2023-11-02T15:35:25.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 34 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:25.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:25.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:25.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter 5 10.d3a59201 osd.0 exist [read 0~256] 2023-11-02T15:35:25.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c127fc640 20 client.4450.objecter 6 10.a7bbaa1 osd.0 noexist [read 0~256] 2023-11-02T15:35:25.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 5 ondisk uv 1 in 10.1 attempt 0 2023-11-02T15:35:25.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:25.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 5 2023-11-02T15:35:25.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 5 2023-11-02T15:35:25.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:25.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 5 2023-11-02T15:35:25.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:25.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 8 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:25.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:25.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:25.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 6 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:25.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:25.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:25.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 6 2023-11-02T15:35:25.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 6 2023-11-02T15:35:25.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:25.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 6 2023-11-02T15:35:25.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:25.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (755 ms) 2023-11-02T15:35:25.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2023-11-02T15:35:25.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:25.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 34 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:25.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:26.360 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:35:23.207+0000 15267640 -1 bdev(0x18e84b30 /var/lib/ceph/osd/ceph-5/block) debug_aio_unlink stalled aio 0x5e1f780 since 2023-11-02T15:35:18.184368+0000, timeout is 5s 2023-11-02T15:35:27.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:27.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:27.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~256] tid 7 osd.0 2023-11-02T15:35:27.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:27.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 7 2023-11-02T15:35:27.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 15 client.4450.objecter _send_op 7 to 10.1 on osd.0 2023-11-02T15:35:27.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:7 10.1 10:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x5632d2da69b0 con 0x5632d2db9280 2023-11-02T15:35:27.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:27.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:27.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cef8a0 2023-11-02T15:35:27.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 34 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:27.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:27.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:27.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:27.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~256] tid 8 osd.0 2023-11-02T15:35:27.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:27.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 8 2023-11-02T15:35:27.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 15 client.4450.objecter _send_op 8 to 10.1 on osd.0 2023-11-02T15:35:27.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:8 10.1 10:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x5632d2cefd90 con 0x5632d2db9280 2023-11-02T15:35:27.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:27.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.007+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:27.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:27.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:27.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:25.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:27.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:26.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:27.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:26.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:27.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:26.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:27.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:26.805+0000 7f7c2387f640 10 client.4450.objecter tick 2023-11-02T15:35:27.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:27.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:28.319 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:27.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:28.319 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:27.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:28.320 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 9 ==== osd_map(35..36 src has 1..36) v4 ==== 951+0+0 (crc 0 0 0) 0x7f7c1c010860 con 0x5632d2db9280 2023-11-02T15:35:28.320 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(35..36 src has 1..36) v4 2023-11-02T15:35:28.320 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 10 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:28.320 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [35,36] > 34 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 35 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 10 client.4450.objecter checking op 7 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 35 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:28.321 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.322 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 10 client.4450.objecter checking op 8 2023-11-02T15:35:28.322 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 35 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:28.322 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:28.322 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 36 2023-11-02T15:35:28.322 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 10 client.4450.objecter checking op 7 2023-11-02T15:35:28.323 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 36 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:28.323 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.323 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 10 client.4450.objecter checking op 8 2023-11-02T15:35:28.323 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 36 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter 7 10.d3a59201 osd.0 exist [read 0~256] 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c127fc640 20 client.4450.objecter 8 10.a7bbaa1 osd.0 noexist [read 0~256] 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 7 ondisk uv 1 in 10.1 attempt 0 2023-11-02T15:35:28.324 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:28.325 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 7 2023-11-02T15:35:28.325 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 7 2023-11-02T15:35:28.325 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:28.325 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 7 2023-11-02T15:35:28.325 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:28.326 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 11 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:28.326 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:28.326 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:28.326 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 8 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:28.326 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:28.327 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:28.327 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 8 2023-11-02T15:35:28.327 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 8 2023-11-02T15:35:28.327 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.327 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 8 2023-11-02T15:35:28.329 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:28.329 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (3309 ms) 2023-11-02T15:35:28.330 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2023-11-02T15:35:28.330 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-11-02T15:35:28.330 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2da62a0 write_seq 1 2023-11-02T15:35:28.331 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:28.331 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 36 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:28.331 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.332 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:28.332 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.332 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [write 0~5 in=5b] tid 9 osd.0 2023-11-02T15:35:28.332 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.332 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 9 2023-11-02T15:35:28.333 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 15 client.4450.objecter _send_op 9 to 10.1 on osd.0 2023-11-02T15:35:28.333 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:9 10.1 10:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36) v8 -- 0x5632d2cef230 con 0x5632d2db9280 2023-11-02T15:35:28.333 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:28.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.316+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-11-02T15:35:28.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 12 ==== osd_map(37..37 src has 1..37) v4 ==== 628+0+0 (crc 0 0 0) 0x7f7c1c010a20 con 0x5632d2db9280 2023-11-02T15:35:28.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(37..37 src has 1..37) v4 2023-11-02T15:35:28.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [37,37] > 36 2023-11-02T15:35:28.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 37 2023-11-02T15:35:28.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 10 client.4450.objecter checking op 9 2023-11-02T15:35:28.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 37 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:28.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:28.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.359+0000 7f7c127fc640 20 client.4450.objecter 9 10.d3a59201 osd.0 exist [write 0~5 in=5b] 2023-11-02T15:35:28.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 13 ==== osd_op_reply(9 exist [write 0~5] v37'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:28.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(9 exist [write 0~5] v37'2 uv2 ondisk = 0) v8 2023-11-02T15:35:28.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:28.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 9 ondisk uv 2 in 10.1 attempt 0 2023-11-02T15:35:28.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:28.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 9 2023-11-02T15:35:28.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 9 2023-11-02T15:35:28.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 9 2023-11-02T15:35:28.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:28.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2da62a0 2023-11-02T15:35:28.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (68 ms) 2023-11-02T15:35:28.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2023-11-02T15:35:28.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-11-02T15:35:28.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2cef5e0 write_seq 2 2023-11-02T15:35:28.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:28.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 37 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:28.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:28.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [write 0~5 in=5b] tid 10 osd.0 2023-11-02T15:35:28.436 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 10 2023-11-02T15:35:28.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 15 client.4450.objecter _send_op 10 to 10.1 on osd.0 2023-11-02T15:35:28.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:10 10.1 10:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37) v8 -- 0x5632d2cef230 con 0x5632d2db9280 2023-11-02T15:35:28.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.437 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:28.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.384+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-11-02T15:35:28.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 14 ==== osd_op_reply(10 exist [write 0~5] v37'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:28.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(10 exist [write 0~5] v37'3 uv3 ondisk = 0) v8 2023-11-02T15:35:28.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:28.438 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 10 ondisk uv 3 in 10.1 attempt 0 2023-11-02T15:35:28.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:28.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 10 2023-11-02T15:35:28.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 10 2023-11-02T15:35:28.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.439 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 10 2023-11-02T15:35:28.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:28.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2cef5e0 2023-11-02T15:35:28.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (49 ms) 2023-11-02T15:35:28.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2023-11-02T15:35:28.440 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2023-11-02T15:35:28.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2cef450 write_seq 3 2023-11-02T15:35:28.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2da4f10 2023-11-02T15:35:28.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 37 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:28.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.441 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:28.442 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.519 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:40.672 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.588 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:40.684 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.593 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:40.745 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.594 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:40.688 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.594 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:40.748 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.622 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:40.776 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.863 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:02:41.017 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.913 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:02:41.010 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:28.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [write 0~5 in=5b] tid 11 osd.0 2023-11-02T15:35:28.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 11 2023-11-02T15:35:28.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 15 client.4450.objecter _send_op 11 to 10.1 on osd.0 2023-11-02T15:35:28.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:11 10.1 10:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37) v8 -- 0x5632d2cf3110 con 0x5632d2db9280 2023-11-02T15:35:28.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:28.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.433+0000 7f7c24a29780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2023-11-02T15:35:28.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:28.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:28.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:28.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (539 ms) 2023-11-02T15:35:28.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2023-11-02T15:35:28.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 15 ==== osd_op_reply(11 exist [write 0~5] v37'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:28.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(11 exist [write 0~5] v37'4 uv4 ondisk = 0) v8 2023-11-02T15:35:28.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:28.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 11 ondisk uv 4 in 10.1 attempt 0 2023-11-02T15:35:28.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:28.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 11 2023-11-02T15:35:28.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 11 2023-11-02T15:35:28.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:28.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 11 2023-11-02T15:35:28.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:28.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2cef450 2023-11-02T15:35:28.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2aa0 2023-11-02T15:35:28.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 37 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:28.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:28.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:28.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:28.986 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~0] tid 12 osd.0 2023-11-02T15:35:29.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:29.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 12 2023-11-02T15:35:29.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 15 client.4450.objecter _send_op 12 to 10.1 on osd.0 2023-11-02T15:35:29.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:12 10.1 10:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x5632d2cf2f90 con 0x5632d2db9280 2023-11-02T15:35:29.164 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.164 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.164 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf0190 2023-11-02T15:35:29.164 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 37 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.165 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:29.165 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:29.165 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.165 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~0] tid 13 osd.0 2023-11-02T15:35:29.165 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.166 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 13 2023-11-02T15:35:29.166 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 15 client.4450.objecter _send_op 13 to 10.1 on osd.0 2023-11-02T15:35:29.166 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:13 10.1 10:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) v8 -- 0x5632d2cf0680 con 0x5632d2db9280 2023-11-02T15:35:29.166 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.167 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:28.972+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:29.167 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 16 ==== osd_map(38..38 src has 1..38) v4 ==== 707+0+0 (crc 0 0 0) 0x7f7c1c010c30 con 0x5632d2db9280 2023-11-02T15:35:29.167 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 17 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.167 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(38..38 src has 1..38) v4 2023-11-02T15:35:29.167 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-11-02T15:35:29.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [38,38] > 37 2023-11-02T15:35:29.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 38 2023-11-02T15:35:29.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 10 client.4450.objecter checking op 12 2023-11-02T15:35:29.169 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 38 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.169 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:29.172 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 10 client.4450.objecter checking op 13 2023-11-02T15:35:29.173 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 38 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.173 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:29.173 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:29.173 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter 12 10.d3a59201 osd.0 exist [read 0~0] 2023-11-02T15:35:29.173 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c127fc640 20 client.4450.objecter 13 10.a7bbaa1 osd.0 noexist [read 0~0] 2023-11-02T15:35:29.174 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (188 ms) 2023-11-02T15:35:29.174 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2023-11-02T15:35:29.174 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 12 ondisk uv 4 in 10.1 attempt 0 2023-11-02T15:35:29.174 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:29.175 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 12 2023-11-02T15:35:29.175 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 12 2023-11-02T15:35:29.175 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.175 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 12 2023-11-02T15:35:29.176 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.176 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 18 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.176 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:29.176 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 13 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 13 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 13 2023-11-02T15:35:29.177 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 13 2023-11-02T15:35:29.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:29.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2aa0 2023-11-02T15:35:29.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.198 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:02:41.350 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:29.521 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:29.522 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:29.522 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:29.522 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~0] tid 14 osd.0 2023-11-02T15:35:29.522 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:29.523 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 14 2023-11-02T15:35:29.523 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 15 client.4450.objecter _send_op 14 to 10.1 on osd.0 2023-11-02T15:35:29.523 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:14 10.1 10:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf2f90 con 0x5632d2db9280 2023-11-02T15:35:29.523 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.524 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.524 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf0190 2023-11-02T15:35:29.524 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.524 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:29.524 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:29.525 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~0] tid 15 osd.0 2023-11-02T15:35:29.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 15 2023-11-02T15:35:29.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 15 client.4450.objecter _send_op 15 to 10.1 on osd.0 2023-11-02T15:35:29.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:15 10.1 10:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf0680 con 0x5632d2db9280 2023-11-02T15:35:29.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.160+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:29.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 19 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-11-02T15:35:29.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 14 ondisk uv 4 in 10.1 attempt 0 2023-11-02T15:35:29.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:29.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 14 2023-11-02T15:35:29.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 14 2023-11-02T15:35:29.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 14 2023-11-02T15:35:29.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.518+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 20 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:29.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 15 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:29.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:29.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:29.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 15 2023-11-02T15:35:29.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 15 2023-11-02T15:35:29.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 15 2023-11-02T15:35:29.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:29.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (358 ms) 2023-11-02T15:35:29.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2023-11-02T15:35:29.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf0190 2023-11-02T15:35:29.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base exist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:29.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:29.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:29.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [read 0~0] tid 16 osd.0 2023-11-02T15:35:29.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:29.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 16 2023-11-02T15:35:29.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 15 client.4450.objecter _send_op 16 to 10.1 on osd.0 2023-11-02T15:35:29.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:16 10.1 10:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf0680 con 0x5632d2db9280 2023-11-02T15:35:29.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2e40 2023-11-02T15:35:29.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base noexist @10 precalc_pgid 0 pgid 0.0 is_read 2023-11-02T15:35:29.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _calc_target target noexist @10 -> pgid 10.a7bbaa1 2023-11-02T15:35:29.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.a7bbaa1 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:29.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid noexist '@10' '@10' [read 0~0] tid 17 osd.0 2023-11-02T15:35:29.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:29.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 17 2023-11-02T15:35:29.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 15 client.4450.objecter _send_op 17 to 10.1 on osd.0 2023-11-02T15:35:29.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:17 10.1 10:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf3680 con 0x5632d2db9280 2023-11-02T15:35:29.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.519+0000 7f7c24a29780 5 client.4450.objecter 2 in flight 2023-11-02T15:35:29.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 21 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2023-11-02T15:35:29.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 16 ondisk uv 4 in 10.1 attempt 0 2023-11-02T15:35:29.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 5 2023-11-02T15:35:29.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 16 2023-11-02T15:35:29.658 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 16 2023-11-02T15:35:29.660 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 5 2023-11-02T15:35:29.660 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 16 2023-11-02T15:35:29.661 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.566+0000 7f7c21e1c640 5 client.4450.objecter 1 in flight 2023-11-02T15:35:29.661 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 22 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:29.661 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2023-11-02T15:35:29.661 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:29.662 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 17 ondisk uv 0 in 10.1 attempt 0 2023-11-02T15:35:30.217 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2023-11-02T15:35:30.217 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:30.218 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 17 2023-11-02T15:35:30.218 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 17 2023-11-02T15:35:30.218 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:30.218 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 17 2023-11-02T15:35:30.218 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:30.219 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (133 ms) 2023-11-02T15:35:30.219 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2023-11-02T15:35:30.219 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2cf2cf0 write_seq 4 2023-11-02T15:35:30.219 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2e40 2023-11-02T15:35:30.219 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:30.220 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:30.220 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:30.220 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:30.220 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [writefull 0~5 in=5b] tid 18 osd.0 2023-11-02T15:35:30.220 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:30.221 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 18 2023-11-02T15:35:30.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 15 client.4450.objecter _send_op 18 to 10.1 on osd.0 2023-11-02T15:35:30.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:18 10.1 10:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf0a80 con 0x5632d2db9280 2023-11-02T15:35:30.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:30.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.651+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:30.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:30.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:30.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:30.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:29.683+0000 7f7c117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T15:35:30.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 23 ==== osd_op_reply(18 exist [writefull 0~5] v38'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:30.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(18 exist [writefull 0~5] v38'5 uv5 ondisk = 0) v8 2023-11-02T15:35:30.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:30.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 18 ondisk uv 5 in 10.1 attempt 0 2023-11-02T15:35:30.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:30.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 18 2023-11-02T15:35:30.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 18 2023-11-02T15:35:30.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:30.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 18 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2cf2cf0 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (564 ms) 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2cf0e30 write_seq 5 2023-11-02T15:35:30.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2e20 2023-11-02T15:35:30.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 38 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:30.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:30.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:30.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:30.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [writefull 0~5 in=5b] tid 19 osd.0 2023-11-02T15:35:30.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:30.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 19 2023-11-02T15:35:30.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 15 client.4450.objecter _send_op 19 to 10.1 on osd.0 2023-11-02T15:35:30.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:19 10.1 10:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38) v8 -- 0x5632d2cf0a80 con 0x5632d2db9280 2023-11-02T15:35:30.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:30.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.215+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:30.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:30.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:30.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:30.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 24 ==== osd_map(39..39 src has 1..39) v4 ==== 1043+0+0 (crc 0 0 0) 0x7f7c1c010c30 con 0x5632d2db9280 2023-11-02T15:35:30.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(39..39 src has 1..39) v4 2023-11-02T15:35:30.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [39,39] > 38 2023-11-02T15:35:30.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 39 2023-11-02T15:35:31.845 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 10 client.4450.objecter checking op 19 2023-11-02T15:35:31.846 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 39 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:31.846 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:31.846 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:31.846 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:30.828+0000 7f7c127fc640 20 client.4450.objecter 19 10.d3a59201 osd.0 exist [writefull 0~5 in=5b] 2023-11-02T15:35:31.846 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.683+0000 7f7c117fa640 10 monclient: tick 2023-11-02T15:35:31.847 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.683+0000 7f7c117fa640 10 monclient: _check_auth_tickets 2023-11-02T15:35:31.847 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.683+0000 7f7c117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T15:35:31.847 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.806+0000 7f7c2387f640 10 client.4450.objecter tick 2023-11-02T15:35:31.847 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 25 ==== osd_op_reply(19 exist [writefull 0~5] v39'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:31.847 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(19 exist [writefull 0~5] v39'6 uv6 ondisk = 0) v8 2023-11-02T15:35:31.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:31.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 19 ondisk uv 6 in 10.1 attempt 0 2023-11-02T15:35:31.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:31.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 19 2023-11-02T15:35:31.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 19 2023-11-02T15:35:31.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:31.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 19 2023-11-02T15:35:31.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:31.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2cf0e30 2023-11-02T15:35:31.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1627 ms) 2023-11-02T15:35:31.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2023-11-02T15:35:31.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 librados: queue_aio_write 0x5632d2c94dc0 completion 0x5632d2cf0f40 write_seq 6 2023-11-02T15:35:31.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 10 client.4450.objecter _op_submit op 0x5632d2cf2e20 2023-11-02T15:35:31.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 client.4450.objecter _calc_target epoch 39 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:31.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:31.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 10 client.4450.objecter _calc_target raw pgid 10.d3a59201 -> actual 10.1 acting [0,2] primary 0 2023-11-02T15:35:31.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 client.4450.objecter _get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:31.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 10 client.4450.objecter _op_submit oid exist '@10' '@10' [writefull 0~5 in=5b] tid 20 osd.0 2023-11-02T15:35:31.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 client.4450.objecter get_session s=0x5632d2c36d10 osd=0 3 2023-11-02T15:35:31.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 15 client.4450.objecter _session_op_assign 0 20 2023-11-02T15:35:32.681 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 15 client.4450.objecter _send_op 20 to 10.1 on osd.0 2023-11-02T15:35:32.682 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- osd_op(unknown.0.0:20 10.1 10:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39) v8 -- 0x5632d2cf0800 con 0x5632d2db9280 2023-11-02T15:35:32.682 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:32.682 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.842+0000 7f7c24a29780 5 client.4450.objecter 1 in flight 2023-11-02T15:35:32.683 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 26 ==== osd_map(40..40 src has 1..40) v4 ==== 1047+0+0 (crc 0 0 0) 0x7f7c1c010c30 con 0x5632d2db9280 2023-11-02T15:35:32.683 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_map(40..40 src has 1..40) v4 2023-11-02T15:35:32.683 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map got epochs [40,40] > 39 2023-11-02T15:35:32.683 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 3 client.4450.objecter handle_osd_map decoding incremental epoch 40 2023-11-02T15:35:32.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 10 client.4450.objecter checking op 20 2023-11-02T15:35:32.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 20 client.4450.objecter _calc_target epoch 40 base exist @10 precalc_pgid 0 pgid 0.0 is_write 2023-11-02T15:35:32.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 20 client.4450.objecter _calc_target target exist @10 -> pgid 10.d3a59201 2023-11-02T15:35:32.685 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 20 client.4450.objecter dump_active .. 0 homeless 2023-11-02T15:35:32.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:31.971+0000 7f7c127fc640 20 client.4450.objecter 20 10.d3a59201 osd.0 exist [writefull 0~5 in=5b] 2023-11-02T15:35:32.686 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 <== osd.0 v2:172.21.15.174:6802/86149 27 ==== osd_op_reply(20 exist [writefull 0~5] v40'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f7c1c016020 con 0x5632d2db9280 2023-11-02T15:35:32.687 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 10 client.4450.objecter ms_dispatch 0x5632d2c8bb30 osd_op_reply(20 exist [writefull 0~5] v40'7 uv7 ondisk = 0) v8 2023-11-02T15:35:32.688 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 10 client.4450.objecter in handle_osd_op_reply 2023-11-02T15:35:32.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 7 client.4450.objecter handle_osd_op_reply 20 ondisk uv 7 in 10.1 attempt 0 2023-11-02T15:35:32.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 10 client.4450.objecter op 0 rval 0 len 0 2023-11-02T15:35:32.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 15 client.4450.objecter handle_osd_op_reply completed tid 20 2023-11-02T15:35:32.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 15 client.4450.objecter _finish_op 20 2023-11-02T15:35:32.689 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 4 2023-11-02T15:35:32.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 15 client.4450.objecter _session_op_remove 0 20 2023-11-02T15:35:32.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 5 client.4450.objecter 0 in flight 2023-11-02T15:35:32.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 20 librados: complete_aio_write 0x5632d2cf0f40 2023-11-02T15:35:32.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (837 ms) 2023-11-02T15:35:32.690 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 10 librados: watch_flush enter 2023-11-02T15:35:32.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 10 librados: watch_flush exit 2023-11-02T15:35:32.691 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 10 client.4450.objecter close_session for osd.0 2023-11-02T15:35:32.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x5632d2db9280 msgr2=0x5632d2e5f5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:32.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x5632d2db9280 0x5632d2e5f5d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T15:35:32.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 20 client.4450.objecter put_session s=0x5632d2c36d10 osd=0 1 2023-11-02T15:35:32.692 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 20 client.4450.objecter shutdown clearing up homeless session... 2023-11-02T15:35:32.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 10 client.4450.objecter successfully canceled tick 2023-11-02T15:35:32.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f7c0406a030 msgr2=0x7f7c0406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:32.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f7c0406a030 0x7f7c0406c4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7c14001d70 tx=0x7f7c14001370 comp rx=0 tx=0).stop 2023-11-02T15:35:32.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 10 monclient: shutdown 2023-11-02T15:35:32.693 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T15:35:32.694 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 msgr2=0x5632d2db8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T15:35:32.694 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 --2- 172.21.15.174:0/1305488291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632d2dbd230 0x5632d2db8800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801cdb0 tx=0x7f7c1804ac60 comp rx=0 tx=0).stop 2023-11-02T15:35:32.694 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c21e1c640 1 -- 172.21.15.174:0/1305488291 reap_dead start 2023-11-02T15:35:32.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 shutdown_connections 2023-11-02T15:35:32.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 >> 172.21.15.174:0/1305488291 conn(0x5632d2ce2870 msgr2=0x5632d2dc36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T15:35:32.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 shutdown_connections 2023-11-02T15:35:32.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 -- 172.21.15.174:0/1305488291 wait complete. 2023-11-02T15:35:32.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.680+0000 7f7c24a29780 1 librados: shutdown 2023-11-02T15:35:32.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] 12 tests from AsioRados (13251 ms total) 2023-11-02T15:35:32.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2023-11-02T15:35:32.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] Global test environment tear-down 2023-11-02T15:35:32.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (26077 ms total) 2023-11-02T15:35:32.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ PASSED ] 12 tests. 2023-11-02T15:35:38.449 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 56.655% pgs not active 2023-11-02T15:35:38.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 298 unknown 2023-11-02T15:35:38.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 284 creating+peering 2023-11-02T15:35:38.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 197 creating+activating 2023-11-02T15:35:38.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 70 active+clean 2023-11-02T15:35:38.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:38.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: io: 2023-11-02T15:35:38.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: client: 198 B/s wr, 0 op/s rd, 0 op/s wr 2023-11-02T15:35:38.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:38.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:38.452 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6185 ms) 2023-11-02T15:35:38.452 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.Status 2023-11-02T15:35:38.452 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.Status (20070 ms) 2023-11-02T15:35:38.452 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] 4 tests from LibRadosService (31675 ms total) 2023-11-02T15:35:38.452 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2023-11-02T15:35:38.453 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] Global test environment tear-down 2023-11-02T15:35:38.453 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31675 ms total) 2023-11-02T15:35:38.453 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ PASSED ] 4 tests. 2023-11-02T15:35:38.870 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2023-11-02T15:35:38.871 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] Global test environment set-up. 2023-11-02T15:35:38.871 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2023-11-02T15:35:38.871 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2023-11-02T15:35:38.871 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5620 ms) 2023-11-02T15:35:38.871 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (374 ms) 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20684 ms) 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: attempt 0 of 20 2023-11-02T15:35:38.872 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5425 ms) 2023-11-02T15:35:38.873 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (32103 ms total) 2023-11-02T15:35:38.873 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: 2023-11-02T15:35:38.873 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] Global test environment tear-down 2023-11-02T15:35:38.873 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (32103 ms total) 2023-11-02T15:35:38.873 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ PASSED ] 4 tests. 2023-11-02T15:35:40.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2023-11-02T15:35:40.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2023-11-02T15:35:40.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2023-11-02T15:35:40.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2023-11-02T15:35:40.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (33549 ms) 2023-11-02T15:35:40.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (33549 ms total) 2023-11-02T15:35:40.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: 2023-11-02T15:35:40.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2023-11-02T15:35:40.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (33550 ms total) 2023-11-02T15:35:40.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2023-11-02T15:35:46.648 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:46.632+0000 12a62640 -1 osd.1 53 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4423.0:3 4.9 4:9294e480:nspace::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) 2023-11-02T15:35:47.887 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:47.880+0000 12a62640 -1 osd.1 54 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4423.0:3 4.9 4:9294e480:nspace::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) 2023-11-02T15:35:48.956 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:01.110 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:48.958 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:01.112 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:48.978 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:48.909+0000 12a62640 -1 osd.1 55 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4423.0:3 4.9 4:9294e480:nspace::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) 2023-11-02T15:35:49.016 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:01.113 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.023 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:01.116 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.028 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:01.182 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:01.188 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.094 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:01.247 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.145 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:01.241 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:49.160 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:01.313 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:35:50.115 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:49.989+0000 12a62640 -1 osd.1 57 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:51.065 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:51.059+0000 12a62640 -1 osd.1 57 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:52.092 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:52.089+0000 12a62640 -1 osd.1 58 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:53.144 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:53.054+0000 12a62640 -1 osd.1 60 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:54.052 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:54.050+0000 12a62640 -1 osd.1 60 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:55.081 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:55.075+0000 12a62640 -1 osd.1 61 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:56.138 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:56.132+0000 12a62640 -1 osd.1 61 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:56.479 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: Running main() from gmock_main.cc 2023-11-02T15:35:56.480 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2023-11-02T15:35:56.480 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] Global test environment set-up. 2023-11-02T15:35:56.480 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] 9 tests from LibRadosPools 2023-11-02T15:35:56.480 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2023-11-02T15:35:56.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolList (5202 ms) 2023-11-02T15:35:56.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2023-11-02T15:35:56.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4220 ms) 2023-11-02T15:35:56.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2023-11-02T15:35:56.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4262 ms) 2023-11-02T15:35:56.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2023-11-02T15:35:56.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (4246 ms) 2023-11-02T15:35:56.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2023-11-02T15:35:56.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3490 ms) 2023-11-02T15:35:56.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (7263 ms) 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5024 ms) 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5941 ms) 2023-11-02T15:35:56.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2023-11-02T15:35:56.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (10196 ms) 2023-11-02T15:35:56.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] 9 tests from LibRadosPools (49845 ms total) 2023-11-02T15:35:56.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: 2023-11-02T15:35:56.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] Global test environment tear-down 2023-11-02T15:35:56.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (49845 ms total) 2023-11-02T15:35:56.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ PASSED ] 9 tests. 2023-11-02T15:35:57.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:57.135+0000 12a62640 -1 osd.1 62 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:58.156 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:58.153+0000 12a62640 -1 osd.1 63 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:35:59.362 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:35:59.344+0000 12a62640 -1 osd.1 65 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:36:00.368 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:36:00.361+0000 12a62640 -1 osd.1 65 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32:602f83fe:::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:36:01.400 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:36:01.394+0000 12a62640 -1 osd.1 66 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4567.0:2 32.6 32:602f83fe:::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2023-11-02T15:36:09.147 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:21.296 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.150 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:21.303 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.190 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:21.344 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.214 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:21.367 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.228 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:21.324 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.236 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:21.331 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.260 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:21.414 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.337 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:21.488 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:09.342 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:21.436 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:14.017 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:36:13.891+0000 12a62640 -1 osd.0 78 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4377.0:8 11.e 11:733a8631:test-rados-api-smithi174-91737-4::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) 2023-11-02T15:36:15.099 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:36:15.092+0000 12a62640 -1 osd.0 79 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4377.0:8 11.e 11:733a8631:test-rados-api-smithi174-91737-4::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) 2023-11-02T15:36:16.078 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:36:16.073+0000 12a62640 -1 osd.0 80 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4377.0:8 11.e 11:733a8631:test-rados-api-smithi174-91737-4::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) 2023-11-02T15:36:17.087 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:36:17.084+0000 12a62640 -1 osd.0 80 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4377.0:8 11.e 11:733a8631:test-rados-api-smithi174-91737-4::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) 2023-11-02T15:36:29.586 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:41.739 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.602 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:41.756 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.608 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:41.704 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.616 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:41.712 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.617 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:41.770 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.622 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:41.774 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.780 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:03:41.877 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.794 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:03:41.944 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:29.835 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:03:41.988 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:36:32.737 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: Running main() from gmock_main.cc 2023-11-02T15:36:32.738 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2023-11-02T15:36:32.738 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] Global test environment set-up. 2023-11-02T15:36:32.738 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2023-11-02T15:36:32.738 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2023-11-02T15:36:32.739 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2023-11-02T15:36:32.739 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2023-11-02T15:36:32.739 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: 2023-11-02T15:36:32.739 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2023-11-02T15:36:32.739 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2023-11-02T15:36:32.740 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (25421 ms) 2023-11-02T15:36:32.740 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2023-11-02T15:36:32.740 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (9261 ms) 2023-11-02T15:36:32.740 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2023-11-02T15:36:32.740 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (8488 ms) 2023-11-02T15:36:32.741 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2023-11-02T15:36:32.741 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (9977 ms) 2023-11-02T15:36:32.741 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2023-11-02T15:36:32.741 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (5276 ms) 2023-11-02T15:36:32.741 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2023-11-02T15:36:32.742 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (6992 ms) 2023-11-02T15:36:32.749 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2023-11-02T15:36:32.749 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (20544 ms) 2023-11-02T15:36:32.750 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (85959 ms total) 2023-11-02T15:36:32.750 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: 2023-11-02T15:36:32.751 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] Global test environment tear-down 2023-11-02T15:36:32.751 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (85959 ms total) 2023-11-02T15:36:32.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2023-11-02T15:36:39.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2023-11-02T15:36:39.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] Global test environment set-up. 2023-11-02T15:36:39.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2023-11-02T15:36:39.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2023-11-02T15:36:39.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2023-11-02T15:36:39.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-11-02T15:36:39.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:36:39.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2023-11-02T15:36:39.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2023-11-02T15:36:39.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2023-11-02T15:36:39.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:35:06.992+0000 7f980a8af780 -1 failed for service _ceph-mon._tcp 2023-11-02T15:36:39.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:35:06.992+0000 7f980a8af780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2023-11-02T15:36:39.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (407 ms) 2023-11-02T15:36:39.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2023-11-02T15:36:39.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2010 ms) 2023-11-02T15:36:39.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2417 ms total) 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743ecce30 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e7432bc740 2023-11-02T15:36:39.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e7434fd410 2023-11-02T15:36:39.936 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743632070 2023-11-02T15:36:39.936 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743632b10 2023-11-02T15:36:39.936 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e74358e4b0 2023-11-02T15:36:39.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743634030 2023-11-02T15:36:39.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743634bc0 2023-11-02T15:36:39.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743635750 2023-11-02T15:36:39.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e7436362e0 2023-11-02T15:36:39.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743636e70 2023-11-02T15:36:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743637a00 2023-11-02T15:36:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743635ee0 2023-11-02T15:36:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743636050 2023-11-02T15:36:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743635350 2023-11-02T15:36:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e7436355f0 2023-11-02T15:36:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743636bd0 2023-11-02T15:36:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x55e743f7b630 2023-11-02T15:36:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 18 aios 2023-11-02T15:36:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743ecce30 2023-11-02T15:36:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e7432bc740 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e7434fd410 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743632070 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743632b10 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e74358e4b0 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743634030 2023-11-02T15:36:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743634bc0 2023-11-02T15:36:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743635750 2023-11-02T15:36:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e7436362e0 2023-11-02T15:36:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743636e70 2023-11-02T15:36:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743637a00 2023-11-02T15:36:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743635ee0 2023-11-02T15:36:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743636050 2023-11-02T15:36:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743635350 2023-11-02T15:36:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e7436355f0 2023-11-02T15:36:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743636bd0 2023-11-02T15:36:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x55e743f7b630 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: done. 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (23168 ms) 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (23168 ms total) 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2023-11-02T15:36:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2023-11-02T15:36:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2023-11-02T15:36:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2023-11-02T15:36:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.Exec (3039 ms) 2023-11-02T15:36:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2023-11-02T15:36:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (61901 ms) 2023-11-02T15:36:40.603 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] L api_stat: Running main() from gmock_main.cc 2023-11-02T15:36:40.603 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2023-11-02T15:36:40.603 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] Global test environment set-up. 2023-11-02T15:36:40.603 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 5 tests from LibRadosStat 2023-11-02T15:36:40.604 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2023-11-02T15:36:40.604 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.Stat (21565 ms) 2023-11-02T15:36:40.604 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2023-11-02T15:36:40.604 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (3416 ms) 2023-11-02T15:36:40.604 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2023-11-02T15:36:40.605 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.StatNS (7731 ms) 2023-11-02T15:36:40.605 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2023-11-02T15:36:40.605 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (267 ms) 2023-11-02T15:36:40.605 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2023-11-02T15:36:40.605 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (559 ms) 2023-11-02T15:36:40.606 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 5 tests from LibRadosStat (33538 ms total) 2023-11-02T15:36:40.606 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: 2023-11-02T15:36:40.607 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2023-11-02T15:36:40.607 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2023-11-02T15:36:40.607 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (40712 ms) 2023-11-02T15:36:40.607 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2023-11-02T15:36:40.607 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (7803 ms) 2023-11-02T15:36:40.608 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2023-11-02T15:36:40.608 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (13 ms) 2023-11-02T15:36:40.608 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2023-11-02T15:36:40.608 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1112 ms) 2023-11-02T15:36:40.608 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (49640 ms total) 2023-11-02T15:36:40.609 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: 2023-11-02T15:36:40.609 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] Global test environment tear-down 2023-11-02T15:36:40.609 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (93936 ms total) 2023-11-02T15:36:40.609 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ PASSED ] 9 tests. 2023-11-02T15:36:43.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: Running main() from gmock_main.cc 2023-11-02T15:36:43.951 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2023-11-02T15:36:43.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] Global test environment set-up. 2023-11-02T15:36:43.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2023-11-02T15:36:43.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2023-11-02T15:36:43.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify_test_cb 2023-11-02T15:36:43.952 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (28204 ms) 2023-11-02T15:36:43.953 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2023-11-02T15:36:43.953 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-11-02T15:36:43.953 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93865231355936 err -107 2023-11-02T15:36:43.953 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (5512 ms) 2023-11-02T15:36:43.953 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2023-11-02T15:36:43.954 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2023-11-02T15:36:43.954 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93865231368576 err -107 2023-11-02T15:36:43.954 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (15993 ms) 2023-11-02T15:36:43.954 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2023-11-02T15:36:43.954 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 292057776128 cookie 93865231369840 2023-11-02T15:36:43.955 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (2804 ms) 2023-11-02T15:36:43.955 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2023-11-02T15:36:43.955 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 296352743424 cookie 93865231369840 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (1951 ms) 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 300647710720 cookie 93865231408016 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (1305 ms) 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2023-11-02T15:36:43.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 313532612610 cookie 93865231403200 2023-11-02T15:36:43.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 313532612610 cookie 93865231408016 2023-11-02T15:36:43.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (4523 ms) 2023-11-02T15:36:43.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2023-11-02T15:36:43.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 330712481793 cookie 93865231408016 2023-11-02T15:36:43.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 330712481793 cookie 93865231408016 2023-11-02T15:36:43.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 335007449090 cookie 93865231408016 2023-11-02T15:36:43.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 339302416387 cookie 93865231408016 2023-11-02T15:36:43.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (5967 ms) 2023-11-02T15:36:43.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2023-11-02T15:36:43.959 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2023-11-02T15:36:43.959 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93865231432176 err -107 2023-11-02T15:36:43.959 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 4398 notify_id 377957122052 cookie 93865231432176 2023-11-02T15:36:43.959 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (7993 ms) 2023-11-02T15:36:43.959 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93865231408016 err -107 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (2181 ms) 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (76433 ms total) 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: 2023-11-02T15:36:43.960 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2023-11-02T15:36:43.961 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2023-11-02T15:36:43.961 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify_test_cb 2023-11-02T15:36:43.961 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (9583 ms) 2023-11-02T15:36:43.961 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (9583 ms total) 2023-11-02T15:36:43.962 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: 2023-11-02T15:36:43.962 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] Global test environment tear-down 2023-11-02T15:36:43.962 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (97262 ms total) 2023-11-02T15:36:43.962 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ PASSED ] 11 tests. 2023-11-02T15:36:44.663 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: Running main() from gmock_main.cc 2023-11-02T15:36:44.664 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2023-11-02T15:36:44.664 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] Global test environment set-up. 2023-11-02T15:36:44.664 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2023-11-02T15:36:44.664 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2023-11-02T15:36:44.665 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (656 ms) 2023-11-02T15:36:44.665 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2023-11-02T15:36:44.665 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (78 ms) 2023-11-02T15:36:44.665 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2023-11-02T15:36:44.665 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (97203 ms) 2023-11-02T15:36:44.666 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (97937 ms total) 2023-11-02T15:36:44.666 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: 2023-11-02T15:36:44.666 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] Global test environment tear-down 2023-11-02T15:36:44.666 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (97937 ms total) 2023-11-02T15:36:44.666 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2023-11-02T15:36:45.928 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: Running main() from gmock_main.cc 2023-11-02T15:36:45.929 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2023-11-02T15:36:45.929 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] Global test environment set-up. 2023-11-02T15:36:45.929 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2023-11-02T15:36:45.929 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: seed 91820 2023-11-02T15:36:45.930 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2023-11-02T15:36:45.930 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (13739 ms) 2023-11-02T15:36:45.930 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2023-11-02T15:36:45.930 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (19894 ms) 2023-11-02T15:36:45.930 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2023-11-02T15:36:45.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (2 ms) 2023-11-02T15:36:45.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2023-11-02T15:36:45.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (41332 ms) 2023-11-02T15:36:45.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2023-11-02T15:36:45.931 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (1994 ms) 2023-11-02T15:36:45.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (76961 ms total) 2023-11-02T15:36:45.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: 2023-11-02T15:36:45.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2023-11-02T15:36:45.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2023-11-02T15:36:45.932 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (4508 ms) 2023-11-02T15:36:45.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2023-11-02T15:36:45.933 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (3 ms) 2023-11-02T15:36:45.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2023-11-02T15:36:45.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (1517 ms) 2023-11-02T15:36:45.934 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2023-11-02T15:36:45.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (5850 ms) 2023-11-02T15:36:45.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (11878 ms total) 2023-11-02T15:36:45.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: 2023-11-02T15:36:45.935 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] Global test environment tear-down 2023-11-02T15:36:45.936 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (99251 ms total) 2023-11-02T15:36:45.936 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ PASSED ] 9 tests. 2023-11-02T15:36:53.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2023-11-02T15:36:53.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2023-11-02T15:36:53.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2023-11-02T15:36:53.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2023-11-02T15:36:53.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2023-11-02T15:36:53.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2023-11-02T15:36:53.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (31429 ms) 2023-11-02T15:36:53.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2023-11-02T15:36:53.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (24786 ms) 2023-11-02T15:36:53.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (56215 ms total) 2023-11-02T15:36:53.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: 2023-11-02T15:36:53.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2023-11-02T15:36:53.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2023-11-02T15:36:53.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2023-11-02T15:36:53.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (4207 ms) 2023-11-02T15:36:53.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2023-11-02T15:36:53.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2023-11-02T15:36:53.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (9458 ms) 2023-11-02T15:36:53.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2023-11-02T15:36:53.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (935 ms) 2023-11-02T15:36:53.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2023-11-02T15:36:53.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (18881 ms) 2023-11-02T15:36:53.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2023-11-02T15:36:53.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497149170016 notify_id 481036337152 notifier_gid 4774 2023-11-02T15:36:53.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (3164 ms) 2023-11-02T15:36:53.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2023-11-02T15:36:53.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497149175712 notify_id 481036337153 notifier_gid 4774 2023-11-02T15:36:53.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (463 ms) 2023-11-02T15:36:53.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2023-11-02T15:36:53.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 481036337153 notifier_gid 4774 2023-11-02T15:36:53.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (375 ms) 2023-11-02T15:36:53.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2023-11-02T15:36:53.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 485331304449 notifier_gid 4774 2023-11-02T15:36:53.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (1223 ms) 2023-11-02T15:36:53.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2023-11-02T15:36:53.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 485331304450 notifier_gid 4774 2023-11-02T15:36:53.973 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (183 ms) 2023-11-02T15:36:53.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2023-11-02T15:36:53.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 485331304450 notifier_gid 4774 2023-11-02T15:36:53.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (347 ms) 2023-11-02T15:36:53.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2023-11-02T15:36:53.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: trying... 2023-11-02T15:36:53.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 485331304451 notifier_gid 4774 2023-11-02T15:36:53.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: timed out 2023-11-02T15:37:02.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flu api_list: [==========] Running 11 tests from 3 test suites. 2023-11-02T15:37:02.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] Global test environment set-up. 2023-11-02T15:37:02.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 7 tests from LibRadosList 2023-11-02T15:37:02.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2023-11-02T15:37:02.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjects (67397 ms) 2023-11-02T15:37:02.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2023-11-02T15:37:02.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (29317 ms) 2023-11-02T15:37:02.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2023-11-02T15:37:02.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo2,foo3 2023-11-02T15:37:02.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2023-11-02T15:37:02.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo2 2023-11-02T15:37:02.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo3 2023-11-02T15:37:02.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo4,foo5 2023-11-02T15:37:02.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo4 2023-11-02T15:37:02.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo5 2023-11-02T15:37:02.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2023-11-02T15:37:02.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo6,foo7 2023-11-02T15:37:02.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo7 2023-11-02T15:37:02.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo6 2023-11-02T15:37:02.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-11-02T15:37:02.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo4 2023-11-02T15:37:02.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo5 2023-11-02T15:37:02.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo7 2023-11-02T15:37:02.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo6 2023-11-02T15:37:02.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo1 2023-11-02T15:37:02.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo1 2023-11-02T15:37:02.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo2 2023-11-02T15:37:02.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo3 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (7345 ms) 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 1 0 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 10 0 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 13 0 2023-11-02T15:37:02.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 7 0 2023-11-02T15:37:02.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 14 0 2023-11-02T15:37:02.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 0 0 2023-11-02T15:37:02.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 15 0 2023-11-02T15:37:02.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 11 0 2023-11-02T15:37:02.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 5 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 8 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 6 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 3 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 4 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 12 0 2023-11-02T15:37:02.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 9 0 2023-11-02T15:37:02.198 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2 0 2023-11-02T15:37:02.198 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-11-02T15:37:02.198 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (3776 ms) 2023-11-02T15:37:02.198 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2023-11-02T15:37:02.198 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: x cursor=MIN 2023-11-02T15:37:02.199 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=1 cursor=5:02547ec2:::1:head 2023-11-02T15:37:02.199 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=10 cursor=5:52ea6a34:::10:head 2023-11-02T15:37:02.199 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=13 cursor=5:566253c9:::13:head 2023-11-02T15:37:02.199 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=7 cursor=5:5c6b0b28:::7:head 2023-11-02T15:37:02.199 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=14 cursor=5:62a1935d:::14:head 2023-11-02T15:37:02.200 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=0 cursor=5:6cac518f:::0:head 2023-11-02T15:37:02.200 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=15 cursor=5:863748b0:::15:head 2023-11-02T15:37:02.200 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=11 cursor=5:89d3ae78:::11:head 2023-11-02T15:37:02.202 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=5 cursor=5:b29083e3:::5:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=8 cursor=5:bd63b0f1:::8:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=6 cursor=5:c4fdafeb:::6:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=3 cursor=5:cfc208b3:::3:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=4 cursor=5:d83876eb:::4:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=12 cursor=5:de5d7c5f:::12:head 2023-11-02T15:37:02.203 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=9 cursor=5:e960b815:::9:head 2023-11-02T15:37:02.204 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=2 cursor=5:f905c69b:::2:head 2023-11-02T15:37:02.204 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: FIRST> seek to MIN oid=1 2023-11-02T15:37:02.204 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=1 cursor=5:02547ec2:::1:head 2023-11-02T15:37:02.204 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:02547ec2:::1:head 2023-11-02T15:37:02.204 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:02547ec2:::1:head -> 1 2023-11-02T15:37:02.205 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=10 cursor=5:52ea6a34:::10:head 2023-11-02T15:37:02.205 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:52ea6a34:::10:head 2023-11-02T15:37:02.205 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:52ea6a34:::10:head -> 10 2023-11-02T15:37:02.205 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=13 cursor=5:566253c9:::13:head 2023-11-02T15:37:02.205 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:566253c9:::13:head 2023-11-02T15:37:02.206 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:566253c9:::13:head -> 13 2023-11-02T15:37:02.206 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=7 cursor=5:5c6b0b28:::7:head 2023-11-02T15:37:02.206 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:5c6b0b28:::7:head 2023-11-02T15:37:02.206 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:5c6b0b28:::7:head -> 7 2023-11-02T15:37:04.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=14 cursor=5:62a1935d:::14:head 2023-11-02T15:37:04.262 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:62a1935d:::14:head 2023-11-02T15:37:04.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:62a1935d:::14:head -> 14 2023-11-02T15:37:04.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=0 cursor=5:6cac518f:::0:head 2023-11-02T15:37:04.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:6cac518f:::0:head 2023-11-02T15:37:04.263 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:6cac518f:::0:head -> 0 2023-11-02T15:37:04.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=15 cursor=5:863748b0:::15:head 2023-11-02T15:37:04.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:863748b0:::15:head 2023-11-02T15:37:04.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:863748b0:::15:head -> 15 2023-11-02T15:37:04.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=11 cursor=5:89d3ae78:::11:head 2023-11-02T15:37:04.264 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:89d3ae78:::11:head 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:89d3ae78:::11:head -> 11 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=5 cursor=5:b29083e3:::5:head 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:b29083e3:::5:head 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:b29083e3:::5:head -> 5 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=8 cursor=5:bd63b0f1:::8:head 2023-11-02T15:37:04.265 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:bd63b0f1:::8:head 2023-11-02T15:37:04.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:bd63b0f1:::8:head -> 8 2023-11-02T15:37:04.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=6 cursor=5:c4fdafeb:::6:head 2023-11-02T15:37:04.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:c4fdafeb:::6:head 2023-11-02T15:37:04.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:c4fdafeb:::6:head -> 6 2023-11-02T15:37:04.266 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=3 cursor=5:cfc208b3:::3:head 2023-11-02T15:37:04.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:cfc208b3:::3:head 2023-11-02T15:37:04.267 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:cfc208b3:::3:head -> 3 2023-11-02T15:37:04.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=4 cursor=5:d83876eb:::4:head 2023-11-02T15:37:04.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:d83876eb:::4:head 2023-11-02T15:37:04.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:d83876eb:::4:head -> 4 2023-11-02T15:37:04.268 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=12 cursor=5:de5d7c5f:::12:head 2023-11-02T15:37:04.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:de5d7c5f:::12:head 2023-11-02T15:37:04.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:de5d7c5f:::12:head -> 12 2023-11-02T15:37:04.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=9 cursor=5:e960b815:::9:head 2023-11-02T15:37:04.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:e960b815:::9:head 2023-11-02T15:37:04.269 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:e960b815:::9:head -> 9 2023-11-02T15:37:04.270 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=2 cursor=5:f905c69b:::2:head 2023-11-02T15:37:04.270 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:f905c69b:::2:head 2023-11-02T15:37:04.272 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:f905c69b:::2:head -> 2 2023-11-02T15:37:04.272 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:52ea6a34:::10:head 2023-11-02T15:37:04.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:52ea6a34:::10:head expected=5:52ea6a34:::10:head 2023-11-02T15:37:04.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:52ea6a34:::10:head -> 10 2023-11-02T15:37:04.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=10 expected=10 2023-11-02T15:37:04.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:e960b815:::9:head 2023-11-02T15:37:04.273 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:e960b815:::9:head expected=5:e960b815:::9:head 2023-11-02T15:37:04.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:e960b815:::9:head -> 9 2023-11-02T15:37:04.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=9 expected=9 2023-11-02T15:37:04.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:de5d7c5f:::12:head 2023-11-02T15:37:04.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:de5d7c5f:::12:head expected=5:de5d7c5f:::12:head 2023-11-02T15:37:04.274 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:de5d7c5f:::12:head -> 12 2023-11-02T15:37:04.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=12 expected=12 2023-11-02T15:37:04.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:d83876eb:::4:head 2023-11-02T15:37:04.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:d83876eb:::4:head expected=5:d83876eb:::4:head 2023-11-02T15:37:04.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:d83876eb:::4:head -> 4 2023-11-02T15:37:04.275 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=4 expected=4 2023-11-02T15:37:04.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:cfc208b3:::3:head 2023-11-02T15:37:04.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:cfc208b3:::3:head expected=5:cfc208b3:::3:head 2023-11-02T15:37:04.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:cfc208b3:::3:head -> 3 2023-11-02T15:37:04.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=3 expected=3 2023-11-02T15:37:04.276 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:c4fdafeb:::6:head 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:c4fdafeb:::6:head expected=5:c4fdafeb:::6:head 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:c4fdafeb:::6:head -> 6 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=6 expected=6 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:bd63b0f1:::8:head 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:bd63b0f1:::8:head expected=5:bd63b0f1:::8:head 2023-11-02T15:37:04.277 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:bd63b0f1:::8:head -> 8 2023-11-02T15:37:04.289 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=8 expected=8 2023-11-02T15:37:04.289 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:b29083e3:::5:head 2023-11-02T15:37:04.289 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:b29083e3:::5:head expected=5:b29083e3:::5:head 2023-11-02T15:37:04.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:b29083e3:::5:head -> 5 2023-11-02T15:37:04.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=5 expected=5 2023-11-02T15:37:04.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:89d3ae78:::11:head 2023-11-02T15:37:06.644 INFO:tasks.workunit.client.0.smithi174.stdout: a api_c_read_operations: Running main() from gmock_main.cc 2023-11-02T15:37:06.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2023-11-02T15:37:06.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] Global test environment set-up. 2023-11-02T15:37:06.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2023-11-02T15:37:06.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2023-11-02T15:37:06.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2023-11-02T15:37:06.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2023-11-02T15:37:06.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (12852 ms) 2023-11-02T15:37:06.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2023-11-02T15:37:06.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (22601 ms) 2023-11-02T15:37:06.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2023-11-02T15:37:06.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (4040 ms) 2023-11-02T15:37:06.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2023-11-02T15:37:06.647 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (36353 ms) 2023-11-02T15:37:06.647 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2023-11-02T15:37:06.647 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (19771 ms) 2023-11-02T15:37:06.647 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2023-11-02T15:37:06.651 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (3278 ms) 2023-11-02T15:37:06.652 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2023-11-02T15:37:06.652 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (175 ms) 2023-11-02T15:37:06.652 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2023-11-02T15:37:06.653 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (216 ms) 2023-11-02T15:37:06.653 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2023-11-02T15:37:06.653 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (114 ms) 2023-11-02T15:37:06.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2023-11-02T15:37:06.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (98 ms) 2023-11-02T15:37:06.654 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2023-11-02T15:37:06.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (165 ms) 2023-11-02T15:37:06.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2023-11-02T15:37:06.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (147 ms) 2023-11-02T15:37:06.655 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2023-11-02T15:37:06.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (2218 ms) 2023-11-02T15:37:06.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2023-11-02T15:37:06.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (449 ms) 2023-11-02T15:37:06.656 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2023-11-02T15:37:06.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (302 ms) 2023-11-02T15:37:06.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2023-11-02T15:37:06.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (128 ms) 2023-11-02T15:37:06.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (102907 ms total) 2023-11-02T15:37:06.657 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: 2023-11-02T15:37:06.658 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] Global test environment tear-down 2023-11-02T15:37:06.658 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (119873 ms total) 2023-11-02T15:37:06.659 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2023-11-02T15:37:19.522 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: starting. 2023-11-02T15:37:19.523 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: creating pool ceph_test_rados_list_parallel.smithi174-92076 2023-11-02T15:37:19.523 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: created object 0... 2023-11-02T15:37:19.523 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: created object 25... 2023-11-02T15:37:19.523 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: created object 49... 2023-11-02T15:37:19.523 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: finishing. 2023-11-02T15:37:19.524 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[92136]: shutting down. 2023-11-02T15:37:19.524 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: starting. 2023-11-02T15:37:19.524 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: listing objects. 2023-11-02T15:37:19.524 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: listed object 0... 2023-11-02T15:37:19.524 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: listed object 25... 2023-11-02T15:37:19.525 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: saw 50 objects 2023-11-02T15:37:19.525 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[92137]: shutting down. 2023-11-02T15:37:19.525 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.525 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: starting. 2023-11-02T15:37:19.525 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: creating pool ceph_test_rados_list_parallel.smithi174-92076 2023-11-02T15:37:19.526 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: created object 0... 2023-11-02T15:37:19.526 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: created object 25... 2023-11-02T15:37:19.526 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: created object 49... 2023-11-02T15:37:19.526 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: finishing. 2023-11-02T15:37:19.527 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[101709]: shutting down. 2023-11-02T15:37:19.527 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.527 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: starting. 2023-11-02T15:37:19.528 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: listing objects. 2023-11-02T15:37:19.528 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: listed object 0... 2023-11-02T15:37:19.528 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: listed object 25... 2023-11-02T15:37:19.528 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: saw 44 objects 2023-11-02T15:37:19.528 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[101710]: shutting down. 2023-11-02T15:37:19.529 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.529 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: starting. 2023-11-02T15:37:19.529 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: removed 25 objects... 2023-11-02T15:37:19.529 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: removed half of the objects 2023-11-02T15:37:19.529 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: removed 50 objects... 2023-11-02T15:37:19.530 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: removed 50 objects 2023-11-02T15:37:19.530 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[101711]: shutting down. 2023-11-02T15:37:19.530 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.530 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.530 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: starting. 2023-11-02T15:37:19.531 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: creating pool ceph_test_rados_list_parallel.smithi174-92076 2023-11-02T15:37:19.531 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: created object 0... 2023-11-02T15:37:19.531 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: created object 25... 2023-11-02T15:37:19.531 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: created object 49... 2023-11-02T15:37:19.531 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: finishing. 2023-11-02T15:37:19.532 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[108744]: shutting down. 2023-11-02T15:37:19.532 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.532 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.532 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: starting. 2023-11-02T15:37:19.532 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: listing objects. 2023-11-02T15:37:19.533 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: listed object 0... 2023-11-02T15:37:19.533 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: listed object 25... 2023-11-02T15:37:19.533 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: listed object 50... 2023-11-02T15:37:19.533 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: saw 53 objects 2023-11-02T15:37:19.534 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[108745]: shutting down. 2023-11-02T15:37:19.534 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.538 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.538 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: starting. 2023-11-02T15:37:19.539 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: added 25 objects... 2023-11-02T15:37:19.545 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: added half of the objects 2023-11-02T15:37:19.545 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: added 50 objects... 2023-11-02T15:37:19.546 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: added 50 objects 2023-11-02T15:37:19.546 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[108746]: shutting down. 2023-11-02T15:37:19.546 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:19.546 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:20.640 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: Running main() from gmock_main.cc 2023-11-02T15:37:20.640 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2023-11-02T15:37:20.641 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] Global test environment set-up. 2023-11-02T15:37:20.641 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2023-11-02T15:37:20.641 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2023-11-02T15:37:20.641 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (889 ms) 2023-11-02T15:37:20.641 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2023-11-02T15:37:20.642 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (84 ms) 2023-11-02T15:37:20.642 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2023-11-02T15:37:20.643 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (106751 ms) 2023-11-02T15:37:20.643 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2023-11-02T15:37:20.643 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:49.823573+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-91512-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-91512-10"}]': finished 2023-11-02T15:37:20.643 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:52.846019+0000 mon.a [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-02T15:37:20.643 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:57.074951+0000 mon.b [INF] from='client.? 172.21.15.174:0/1927709685' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-11-02T15:37:20.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:57.546344+0000 mon.c [INF] from='client.? 172.21.15.174:0/1975851948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-91413-16"}]: dispatch 2023-11-02T15:37:20.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:58.109616+0000 client.admin [INF] onexx 2023-11-02T15:37:20.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:58.125747+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2023-11-02T15:37:20.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:36:58.724958+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-91413-16"}]: dispatch 2023-11-02T15:37:20.644 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:06.643439+0000 mon.c [INF] from='client.? 172.21.15.174:0/1975851948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-91413-16"}]: dispatch 2023-11-02T15:37:20.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:07.591548+0000 mon.c [INF] from='client.4774 172.21.15.174:0/1192968585' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-91883-4"}]: dispatch 2023-11-02T15:37:20.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:08.137710+0000 mon.a [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-02T15:37:20.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:08.451750+0000 mon.b [INF] from='client.? 172.21.15.174:0/1927709685' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-11-02T15:37:20.645 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.331603+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-91373-9","app": "rados","yes_i_really_mean_it": true}]': finished 2023-11-02T15:37:20.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.346805+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-91470-10"}]': finished 2023-11-02T15:37:20.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.349751+0000 mon.a [INF] from='client.? 172.21.15.174:0/4128346539' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-91455-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-11-02T15:37:20.646 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.436343+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-91413-16"}]: dispatch 2023-11-02T15:37:22.771 INFO:tasks.ceph.mon.c.smithi174.stderr:2023-11-02T15:37:22.740+0000 fb3f640 -1 mon.c@2(peon).paxos(paxos updating c 1..410) lease_expire from mon.0 v2:172.21.15.174:3300/0 is 0.478263348s seconds in the past; mons are probably laggy (or possibly clocks are too skewed) 2023-11-02T15:37:26.224 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.47014 api_lock: Running main() from gmock_main.cc 2023-11-02T15:37:26.225 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2023-11-02T15:37:26.225 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] Global test environment set-up. 2023-11-02T15:37:26.225 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLock 2023-11-02T15:37:26.225 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2023-11-02T15:37:26.226 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (7305 ms) 2023-11-02T15:37:26.226 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2023-11-02T15:37:26.226 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockShared (2298 ms) 2023-11-02T15:37:26.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2023-11-02T15:37:26.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (5422 ms) 2023-11-02T15:37:26.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2023-11-02T15:37:26.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (7682 ms) 2023-11-02T15:37:26.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2023-11-02T15:37:26.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (8594 ms) 2023-11-02T15:37:26.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2023-11-02T15:37:26.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.Unlock (8484 ms) 2023-11-02T15:37:26.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2023-11-02T15:37:26.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (14689 ms) 2023-11-02T15:37:26.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2023-11-02T15:37:26.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (2216 ms) 2023-11-02T15:37:26.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLock (56690 ms total) 2023-11-02T15:37:26.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: 2023-11-02T15:37:26.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2023-11-02T15:37:26.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2023-11-02T15:37:26.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (27806 ms) 2023-11-02T15:37:26.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2023-11-02T15:37:26.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5746 ms) 2023-11-02T15:37:26.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2023-11-02T15:37:26.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (5394 ms) 2023-11-02T15:37:26.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2023-11-02T15:37:26.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1282 ms) 2023-11-02T15:37:26.235 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2023-11-02T15:37:26.235 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (269 ms) 2023-11-02T15:37:26.235 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2023-11-02T15:37:26.235 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (1273 ms) 2023-11-02T15:37:26.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2023-11-02T15:37:26.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (577 ms) 2023-11-02T15:37:26.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2023-11-02T15:37:26.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (338 ms) 2023-11-02T15:37:26.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (42685 ms total) 2023-11-02T15:37:26.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: 2023-11-02T15:37:26.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] Global test environment tear-down 2023-11-02T15:37:26.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (139698 ms total) 2023-11-02T15:37:26.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ PASSED ] 16 tests. 2023-11-02T15:37:29.499 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.499 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: starting. 2023-11-02T15:37:29.499 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: creating pool ceph_test_rados_list_parallel.smithi174-92076 2023-11-02T15:37:29.500 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: created object 0... 2023-11-02T15:37:29.500 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: created object 25... 2023-11-02T15:37:29.500 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: created object 49... 2023-11-02T15:37:29.500 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: finishing. 2023-11-02T15:37:29.500 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[118099]: shutting down. 2023-11-02T15:37:29.501 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.501 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.501 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.501 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: starting. 2023-11-02T15:37:29.501 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: listing objects. 2023-11-02T15:37:29.502 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: listed object 0... 2023-11-02T15:37:29.502 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: listed object 25... 2023-11-02T15:37:29.502 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: listed object 50... 2023-11-02T15:37:29.502 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: listed object 75... 2023-11-02T15:37:29.505 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: saw 100 objects 2023-11-02T15:37:29.505 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[118100]: shutting down. 2023-11-02T15:37:29.505 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.505 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.505 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.506 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: starting. 2023-11-02T15:37:29.506 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: added 25 objects... 2023-11-02T15:37:29.506 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: added half of the objects 2023-11-02T15:37:29.506 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: added 50 objects... 2023-11-02T15:37:29.506 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: added 50 objects 2023-11-02T15:37:29.507 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[118105]: shutting down. 2023-11-02T15:37:29.507 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.507 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.507 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.507 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: starting. 2023-11-02T15:37:29.508 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: removed 25 objects... 2023-11-02T15:37:29.508 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: removed half of the objects 2023-11-02T15:37:29.508 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: removed 50 objects... 2023-11-02T15:37:29.508 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: removed 50 objects 2023-11-02T15:37:29.508 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[118106]: shutting down. 2023-11-02T15:37:29.509 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.509 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.509 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.509 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: starting. 2023-11-02T15:37:29.509 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: added 25 objects... 2023-11-02T15:37:29.510 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: added half of the objects 2023-11-02T15:37:29.510 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: added 50 objects... 2023-11-02T15:37:29.510 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: added 50 objects 2023-11-02T15:37:29.510 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[118101]: shutting down. 2023-11-02T15:37:29.510 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.511 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.511 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.511 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.511 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: starting. 2023-11-02T15:37:29.511 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: creating pool ceph_test_rados_list_parallel.smithi174-92076 2023-11-02T15:37:29.512 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: created object 0... 2023-11-02T15:37:29.512 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: created object 25... 2023-11-02T15:37:29.512 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: created object 49... 2023-11-02T15:37:29.512 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: finishing. 2023-11-02T15:37:29.512 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[126653]: shutting down. 2023-11-02T15:37:29.513 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.515 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.515 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.516 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:29.516 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: starting. 2023-11-02T15:37:29.521 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listing objects. 2023-11-02T15:37:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]0+0000 mon.a [INF] from='client.4774 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-91883-4"}]: dispatch 2023-11-02T15:37:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.774431+0000 mon.a [INF] from='client.? 172.21.15.174:0/4128346539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-91455-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-91455-22"}]: dispatch 2023-11-02T15:37:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:09.945156+0000 mon.a [INF] from='client.? 172.21.15.174:0/1061695908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-91391-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-11-02T15:37:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:13.239622+0000 client.admin [INF] twoxx 2023-11-02T15:37:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:13.264383+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2023-11-02T15:37:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:10.287356+0000 mon.b [INF] from='client.? 172.21.15.174:0/554679101' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-91470-10"}]: dispatch 2023-11-02T15:37:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:11.095320+0000 mon.c [INF] from='client.? 172.21.15.174:0/1975851948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-91413-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-11-02T15:37:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:11.667599+0000 mon.c [INF] from='client.? 172.21.15.174:0/3707541560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-91559-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-11-02T15:37:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:14.289454+0000 mon.a [WRN] Health check update: Slow OSD heartbeats on back (longest 12203.186ms) (OSD_SLOW_PING_TIME_BACK) 2023-11-02T15:37:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:14.475689+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-91470-10"}]: dispatch 2023-11-02T15:37:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:15.311409+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-91413-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2023-11-02T15:37:29.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:15.326218+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-91559-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2023-11-02T15:37:29.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:15.460394+0000 mon.a [INF] from='client.4774 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-91883-4"}]': finished 2023-11-02T15:37:29.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:17.421950+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2023-11-02T15:37:29.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:16.948038+0000 mon.c [INF] from='client.4774 172.21.15.174:0/1192968585' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-91883-4", "tierpool": "test-rados-api-smithi174-91883-6"}]: dispatch 2023-11-02T15:37:29.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:16.988920+0000 mon.b [INF] from='client.? 172.21.15.174:0/4073541022' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-91512-10"}]: dispatch 2023-11-02T15:37:29.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:18.166048+0000 mon.a [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-02T15:37:36.991 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:18.503865+0000 mon.a [INF] from='client.? ' e: listed object 0... 2023-11-02T15:37:36.991 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listed object 25... 2023-11-02T15:37:36.991 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listed object 50... 2023-11-02T15:37:36.992 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listed object 75... 2023-11-02T15:37:36.992 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listed object 100... 2023-11-02T15:37:36.992 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: listed object 125... 2023-11-02T15:37:36.992 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: saw 150 objects 2023-11-02T15:37:36.992 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[126654]: shutting down. 2023-11-02T15:37:36.993 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.993 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.993 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.993 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.993 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: starting. 2023-11-02T15:37:36.994 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: added 25 objects... 2023-11-02T15:37:36.994 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: added half of the objects 2023-11-02T15:37:36.994 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: added 50 objects... 2023-11-02T15:37:36.994 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: added 50 objects 2023-11-02T15:37:36.994 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[126655]: shutting down. 2023-11-02T15:37:36.995 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.995 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.995 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.995 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.996 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2023-11-02T15:37:36.997 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******* SUCCESS ********** 2023-11-02T15:37:39.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: Running main() from gmock_main.cc 2023-11-02T15:37:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2023-11-02T15:37:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] Global test environment set-up. 2023-11-02T15:37:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2023-11-02T15:37:39.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: seed 91512 2023-11-02T15:37:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2023-11-02T15:37:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (28758 ms) 2023-11-02T15:37:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2023-11-02T15:37:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (16537 ms) 2023-11-02T15:37:39.940 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2023-11-02T15:37:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (5977 ms) 2023-11-02T15:37:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2023-11-02T15:37:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (5909 ms) 2023-11-02T15:37:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2023-11-02T15:37:39.941 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3682 ms) 2023-11-02T15:37:39.942 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2023-11-02T15:37:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (15580 ms) 2023-11-02T15:37:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2023-11-02T15:37:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (15735 ms) 2023-11-02T15:37:39.943 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2023-11-02T15:37:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (1973 ms) 2023-11-02T15:37:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (94151 ms total) 2023-11-02T15:37:39.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: 2023-11-02T15:37:39.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2023-11-02T15:37:39.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2023-11-02T15:37:39.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (6145 ms) 2023-11-02T15:37:39.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2023-11-02T15:37:39.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (468 ms) 2023-11-02T15:37:39.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2023-11-02T15:37:39.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1176 ms) 2023-11-02T15:37:39.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2023-11-02T15:37:39.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1098 ms) 2023-11-02T15:37:39.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2023-11-02T15:37:39.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (341 ms) 2023-11-02T15:37:39.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2023-11-02T15:37:39.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (291 ms) 2023-11-02T15:37:39.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (261 ms) 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (305 ms) 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (10085 ms total) 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: 2023-11-02T15:37:39.948 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] Global test environment tear-down 2023-11-02T15:37:39.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (153392 ms total) 2023-11-02T15:37:39.949 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ PASSED ] 16 tests. 2023-11-02T15:37:40.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: Running main() from gmock_main.cc 2023-11-02T15:37:40.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2023-11-02T15:37:40.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] Global test environment set-up. 2023-11-02T15:37:40.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2023-11-02T15:37:40.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: seed 91455 2023-11-02T15:37:40.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2023-11-02T15:37:40.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2023-11-02T15:37:40.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2023-11-02T15:37:40.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (42344 ms) 2023-11-02T15:37:40.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2023-11-02T15:37:40.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (8970 ms) 2023-11-02T15:37:40.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2023-11-02T15:37:40.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (12826 ms) 2023-11-02T15:37:40.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2023-11-02T15:37:40.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (1222 ms) 2023-11-02T15:37:40.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2023-11-02T15:37:40.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2196 ms) 2023-11-02T15:37:40.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2023-11-02T15:37:40.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (9392 ms) 2023-11-02T15:37:40.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2023-11-02T15:37:40.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (1070 ms) 2023-11-02T15:37:40.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2023-11-02T15:37:40.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (241 ms) 2023-11-02T15:37:40.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2023-11-02T15:37:40.169 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (148 ms) 2023-11-02T15:37:40.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2023-11-02T15:37:40.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4428 ms) 2023-11-02T15:37:40.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2023-11-02T15:37:40.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (2596 ms) 2023-11-02T15:37:40.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2023-11-02T15:37:40.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (2370 ms) 2023-11-02T15:37:40.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2023-11-02T15:37:40.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (1409 ms) 2023-11-02T15:37:40.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2023-11-02T15:37:40.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4383 ms) 2023-11-02T15:37:40.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2023-11-02T15:37:40.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (5604 ms) 2023-11-02T15:37:40.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-11-02T15:37:40.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (338 ms) 2023-11-02T15:37:40.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2023-11-02T15:37:40.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (638 ms) 2023-11-02T15:37:40.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2023-11-02T15:37:40.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (162 ms) 2023-11-02T15:37:40.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2023-11-02T15:37:40.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (239 ms) 2023-11-02T15:37:40.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (100578 ms total) 2023-11-02T15:37:40.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: 2023-11-02T15:37:40.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2023-11-02T15:37:40.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2023-11-02T15:37:40.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (10527 ms) 2023-11-02T15:37:40.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2023-11-02T15:37:40.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (1125 ms) 2023-11-02T15:37:40.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2023-11-02T15:37:40.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (211 ms) 2023-11-02T15:37:40.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2023-11-02T15:37:40.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (316 ms) 2023-11-02T15:37:40.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2023-11-02T15:37:41.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ntity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-91512-10"}]: dispatch 2023-11-02T15:37:41.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:20.541264+0000 mon.b [INF] from='client.? 172.21.15.174:0/1927709685' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-11-02T15:37:41.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:23.373528+0000 mon.a [INF] from='client.? 172.21.15.174:0/1061695908' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-91391-7","app": "rados","yes_i_really_mean_it": true}]': finished 2023-11-02T15:37:41.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:23.385777+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-91470-10"}]': finished 2023-11-02T15:37:41.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:23.391614+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-91413-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2023-11-02T15:37:41.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:23.395555+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-91559-9","app": "rados","yes_i_really_mean_it": true}]': finished 2023-11-02T15:37:41.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:24.605178+0000 mon.a [INF] from='client.4774 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-91883-4", "tierpool": "test-rados-api-smithi174-91883-6"}]: dispatch 2023-11-02T15:37:41.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:24.653243+0000 mon.a [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-02T15:37:41.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:24.813713+0000 client.admin [INF] threexx 2023-11-02T15:37:41.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2023-11-02T15:37:25.504300+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2023-11-02T15:37:41.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (46965 ms) 2023-11-02T15:37:41.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (154689 ms total) 2023-11-02T15:37:41.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: 2023-11-02T15:37:41.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] Global test environment tear-down 2023-11-02T15:37:41.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (154689 ms total) 2023-11-02T15:37:41.404 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ PASSED ] 4 tests. 2023-11-02T15:37:48.357 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: starting. 2023-11-02T15:37:48.357 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: creating pool ceph_test_rados_open_pools_parallel.smithi174-92119 2023-11-02T15:37:48.357 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: created object 0... 2023-11-02T15:37:48.358 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: created object 25... 2023-11-02T15:37:48.358 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: created object 49... 2023-11-02T15:37:48.358 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: finishing. 2023-11-02T15:37:48.359 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[92163]: shutting down. 2023-11-02T15:37:48.359 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[92164]: starting. 2023-11-02T15:37:48.360 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[92164]: rados_pool_create. 2023-11-02T15:37:48.360 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[92164]: rados_ioctx_create. 2023-11-02T15:37:48.360 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[92164]: shutting down. 2023-11-02T15:37:48.360 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2023-11-02T15:37:48.360 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: starting. 2023-11-02T15:37:48.362 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: creating pool ceph_test_rados_open_pools_parallel.smithi174-92119 2023-11-02T15:37:48.362 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: created object 0... 2023-11-02T15:37:48.362 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: created object 25... 2023-11-02T15:37:48.362 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: created object 49... 2023-11-02T15:37:48.363 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: finishing. 2023-11-02T15:37:48.363 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[112025]: shutting down. 2023-11-02T15:37:48.363 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2023-11-02T15:37:48.363 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[112026]: starting. 2023-11-02T15:37:48.363 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[112026]: rados_pool_create. 2023-11-02T15:37:48.364 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[112026]: rados_ioctx_create. 2023-11-02T15:37:48.364 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[112026]: shutting down. 2023-11-02T15:37:48.364 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2023-11-02T15:37:48.364 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2023-11-02T15:37:48.364 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******* SUCCESS ********** 2023-11-02T15:37:48.562 INFO:tasks.workunit.client.0.smithi174.stdout:shing 2023-11-02T15:37:48.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushed 2023-11-02T15:37:48.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3358 ms) 2023-11-02T15:37:48.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2023-11-02T15:37:48.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: trying... 2023-11-02T15:37:48.573 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 489626271746 notifier_gid 4774 2023-11-02T15:37:48.574 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: timed out 2023-11-02T15:37:48.574 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2023-11-02T15:37:48.574 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushed 2023-11-02T15:37:48.574 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3217 ms) 2023-11-02T15:37:48.574 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2023-11-02T15:37:48.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: List watches 2023-11-02T15:37:48.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 2023-11-02T15:37:48.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 493921239044 notifier_gid 4774 2023-11-02T15:37:48.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 done 2023-11-02T15:37:48.575 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: watch_check 2023-11-02T15:37:48.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: unwatch2 2023-11-02T15:37:48.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2023-11-02T15:37:48.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: done 2023-11-02T15:37:48.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (4219 ms) 2023-11-02T15:37:48.576 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2023-11-02T15:37:48.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: List watches 2023-11-02T15:37:48.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 2023-11-02T15:37:48.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94497148533760 notify_id 493921239045 notifier_gid 4774 2023-11-02T15:37:48.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 done 2023-11-02T15:37:48.578 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: watch_check 2023-11-02T15:37:48.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: unwatch2 2023-11-02T15:37:48.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2023-11-02T15:37:48.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: done 2023-11-02T15:37:48.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (4015 ms) 2023-11-02T15:37:48.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (54045 ms total) 2023-11-02T15:37:48.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: 2023-11-02T15:37:48.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2023-11-02T15:37:48.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (161862 ms total) 2023-11-02T15:37:48.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2023-11-02T15:37:50.454 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: starting. 2023-11-02T15:37:50.454 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: creating pool ceph_test_rados_delete_pools_parallel.smithi174-92159 2023-11-02T15:37:50.454 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: created object 0... 2023-11-02T15:37:50.455 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: created object 25... 2023-11-02T15:37:50.455 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: created object 49... 2023-11-02T15:37:50.455 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: finishing. 2023-11-02T15:37:50.455 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[92184]: shutting down. 2023-11-02T15:37:50.455 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[92186]: starting. 2023-11-02T15:37:50.456 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[92186]: deleting pool ceph_test_rados_delete_pools_parallel.smithi174-92159 2023-11-02T15:37:50.456 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[92186]: shutting down. 2023-11-02T15:37:50.456 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2023-11-02T15:37:50.456 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: starting. 2023-11-02T15:37:50.456 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: creating pool ceph_test_rados_delete_pools_parallel.smithi174-92159 2023-11-02T15:37:50.457 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: created object 0... 2023-11-02T15:37:50.457 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: created object 25... 2023-11-02T15:37:50.457 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: created object 49... 2023-11-02T15:37:50.457 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: finishing. 2023-11-02T15:37:50.457 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[115702]: shutting down. 2023-11-02T15:37:50.458 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2023-11-02T15:37:50.458 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: starting. 2023-11-02T15:37:50.458 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: listing objects. 2023-11-02T15:37:50.458 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: listed object 0... 2023-11-02T15:37:50.458 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: listed object 25... 2023-11-02T15:37:50.459 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: saw 50 objects 2023-11-02T15:37:50.459 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[115704]: shutting down. 2023-11-02T15:37:50.459 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2023-11-02T15:37:50.459 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[115703]: starting. 2023-11-02T15:37:50.460 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[115703]: deleting pool ceph_test_rados_delete_pools_parallel.smithi174-92159 2023-11-02T15:37:50.461 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[115703]: shutting down. 2023-11-02T15:37:50.463 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2023-11-02T15:37:50.469 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2023-11-02T15:37:50.470 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******* SUCCESS ********** 2023-11-02T15:37:54.013 INFO:tasks.workunit.client.0.smithi174.stdout:ibRadosMisc.CmpExt 2023-11-02T15:37:54.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3397 ms) 2023-11-02T15:37:54.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2023-11-02T15:37:54.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4706 ms) 2023-11-02T15:37:54.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2023-11-02T15:37:54.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2023-11-02T15:37:54.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2023-11-02T15:37:54.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2023-11-02T15:37:54.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2023-11-02T15:37:54.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (65621 ms) 2023-11-02T15:37:54.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 8 tests from LibRadosMisc (138664 ms total) 2023-11-02T15:37:54.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2023-11-02T15:37:54.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] Global test environment tear-down 2023-11-02T15:37:54.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (167414 ms total) 2023-11-02T15:37:54.016 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ PASSED ] 12 tests. 2023-11-02T15:37:54.016 INFO:tasks.workunit.client.0.smithi174.stdout:RoundTripPP2 (134 ms) 2023-11-02T15:37:54.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2023-11-02T15:37:54.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (314 ms) 2023-11-02T15:37:54.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2023-11-02T15:37:54.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (277 ms) 2023-11-02T15:37:54.017 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2023-11-02T15:37:54.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (166 ms) 2023-11-02T15:37:54.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2023-11-02T15:37:54.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (725 ms) 2023-11-02T15:37:54.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2023-11-02T15:37:54.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (604 ms) 2023-11-02T15:37:54.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2023-11-02T15:37:54.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (153 ms) 2023-11-02T15:37:54.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2023-11-02T15:37:54.020 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (171 ms) 2023-11-02T15:37:54.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2023-11-02T15:37:54.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (486 ms) 2023-11-02T15:37:54.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2023-11-02T15:37:54.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (176 ms) 2023-11-02T15:37:54.021 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2023-11-02T15:37:54.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (198 ms) 2023-11-02T15:37:54.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2023-11-02T15:37:54.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (204 ms) 2023-11-02T15:37:54.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2023-11-02T15:37:54.022 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (125 ms) 2023-11-02T15:37:54.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (15912 ms total) 2023-11-02T15:37:54.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: 2023-11-02T15:37:54.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] Global test environment tear-down 2023-11-02T15:37:54.023 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (167498 ms total) 2023-11-02T15:37:54.024 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ PASSED ] 37 tests. 2023-11-02T15:37:55.141 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: Running main() from gmock_main.cc 2023-11-02T15:37:55.142 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2023-11-02T15:37:55.142 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] Global test environment set-up. 2023-11-02T15:37:55.142 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 14 tests from LibRadosIo 2023-11-02T15:37:55.142 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2023-11-02T15:37:55.143 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (26718 ms) 2023-11-02T15:37:55.143 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2023-11-02T15:37:55.143 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2023-11-02T15:37:55.143 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2023-11-02T15:37:55.144 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2023-11-02T15:37:55.144 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2023-11-02T15:37:55.144 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2023-11-02T15:37:55.144 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2023-11-02T15:37:55.144 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2023-11-02T15:37:55.145 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (3259 ms) 2023-11-02T15:37:55.145 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2023-11-02T15:37:55.145 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (2136 ms) 2023-11-02T15:37:55.145 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2023-11-02T15:37:55.145 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.Checksum (1408 ms) 2023-11-02T15:37:55.146 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2023-11-02T15:37:55.146 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (2656 ms) 2023-11-02T15:37:55.147 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2023-11-02T15:37:55.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (2286 ms) 2023-11-02T15:37:55.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2023-11-02T15:37:55.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (12976 ms) 2023-11-02T15:37:55.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2023-11-02T15:37:55.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (327 ms) 2023-11-02T15:37:55.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2023-11-02T15:37:55.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.TruncTest (30979 ms) 2023-11-02T15:37:55.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2023-11-02T15:37:55.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (7802 ms) 2023-11-02T15:37:55.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2023-11-02T15:37:55.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (1615 ms) 2023-11-02T15:37:55.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2023-11-02T15:37:55.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RmXattr (6199 ms) 2023-11-02T15:37:55.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2023-11-02T15:37:55.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.XattrIter (878 ms) 2023-11-02T15:37:55.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 14 tests from LibRadosIo (99239 ms total) 2023-11-02T15:37:55.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: 2023-11-02T15:37:55.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2023-11-02T15:37:55.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2023-11-02T15:37:55.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (4931 ms) 2023-11-02T15:37:55.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2023-11-02T15:37:55.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (256 ms) 2023-11-02T15:37:55.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2023-11-02T15:37:55.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (242 ms) 2023-11-02T15:37:55.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2023-11-02T15:37:55.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (327 ms) 2023-11-02T15:37:55.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2023-11-02T15:37:55.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (439 ms) 2023-11-02T15:37:55.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2023-11-02T15:37:55.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (93 ms) 2023-11-02T15:37:55.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2023-11-02T15:37:55.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (115 ms) 2023-11-02T15:37:55.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2023-11-02T15:37:55.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (107 ms) 2023-11-02T15:37:55.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2023-11-02T15:37:55.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (400 ms) 2023-11-02T15:37:55.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2023-11-02T15:37:55.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (190 ms) 2023-11-02T15:37:55.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 10 tests from LibRadosIoEC (7100 ms total) 2023-11-02T15:37:55.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: 2023-11-02T15:37:55.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] Global test environment tear-down 2023-11-02T15:37:55.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [==========] 24 tests from 2 test suites ran. (168643 ms total) 2023-11-02T15:37:55.168 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ PASSED ] 24 tests. 2023-11-02T15:38:00.437 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:12.591 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.508 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:12.661 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.577 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:12.673 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.581 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:12.734 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.586 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:12.682 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.589 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:12.743 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.684 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:12.838 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.735 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:12.831 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:00.772 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:12.925 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.631 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:32.783 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.695 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:32.791 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.696 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:32.793 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.706 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:32.855 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.770 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:32.863 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.771 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:32.924 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.780 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:32.933 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.781 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:32.935 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:20.854 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:33.008 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:29.827 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: Running main() from gmock_main.cc 2023-11-02T15:38:29.828 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2023-11-02T15:38:29.828 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] Global test environment set-up. 2023-11-02T15:38:29.828 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2023-11-02T15:38:29.828 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2023-11-02T15:38:29.828 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (22544 ms) 2023-11-02T15:38:29.829 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2023-11-02T15:38:29.829 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (24225 ms) 2023-11-02T15:38:29.829 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2023-11-02T15:38:29.829 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (5825 ms) 2023-11-02T15:38:29.829 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2023-11-02T15:38:29.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2970 ms) 2023-11-02T15:38:29.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (55564 ms total) 2023-11-02T15:38:29.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2023-11-02T15:38:29.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2023-11-02T15:38:29.830 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2023-11-02T15:38:29.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (27176 ms) 2023-11-02T15:38:29.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2023-11-02T15:38:29.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (10961 ms) 2023-11-02T15:38:29.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2023-11-02T15:38:29.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (39038 ms) 2023-11-02T15:38:29.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (77175 ms total) 2023-11-02T15:38:29.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2023-11-02T15:38:29.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2023-11-02T15:38:29.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2023-11-02T15:38:29.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (4390 ms) 2023-11-02T15:38:29.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2023-11-02T15:38:29.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (3083 ms) 2023-11-02T15:38:29.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2023-11-02T15:38:29.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (5014 ms) 2023-11-02T15:38:29.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2023-11-02T15:38:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2242 ms) 2023-11-02T15:38:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (14729 ms total) 2023-11-02T15:38:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2023-11-02T15:38:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2023-11-02T15:38:29.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2023-11-02T15:38:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (5649 ms) 2023-11-02T15:38:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2023-11-02T15:38:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (5746 ms) 2023-11-02T15:38:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (11395 ms total) 2023-11-02T15:38:29.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2023-11-02T15:38:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] Global test environment tear-down 2023-11-02T15:38:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (203207 ms total) 2023-11-02T15:38:29.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ PASSED ] 13 tests. 2023-11-02T15:38:40.643 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:52.797 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.646 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:52.800 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.695 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:05:52.849 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.696 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:52.850 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.707 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:52.855 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.707 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:52.799 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.708 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:52.802 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.750 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:05:52.903 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:40.755 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:05:52.851 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:38:52.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2023-11-02T15:38:52.848 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [==========] Running 19 tests from 4 test suites. 2023-11-02T15:38:52.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2023-11-02T15:38:52.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2023-11-02T15:38:52.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: seed 91737 2023-11-02T15:38:52.849 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2023-11-02T15:38:52.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (20624 ms) 2023-11-02T15:38:52.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2023-11-02T15:38:52.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (13728 ms) 2023-11-02T15:38:52.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2023-11-02T15:38:52.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (42361 ms) 2023-11-02T15:38:52.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2023-11-02T15:38:52.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3146 ms) 2023-11-02T15:38:52.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2023-11-02T15:38:52.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (10354 ms) 2023-11-02T15:38:52.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (90214 ms total) 2023-11-02T15:38:52.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2023-11-02T15:38:52.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2023-11-02T15:38:52.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2023-11-02T15:38:52.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (16628 ms) 2023-11-02T15:38:52.855 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2023-11-02T15:38:52.855 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (51858 ms) 2023-11-02T15:38:52.855 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2023-11-02T15:38:52.855 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (11424 ms) 2023-11-02T15:38:52.855 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2023-11-02T15:38:52.856 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (7214 ms) 2023-11-02T15:38:52.856 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2023-11-02T15:38:52.856 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2482 ms) 2023-11-02T15:38:52.856 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-11-02T15:38:52.856 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.0-1159-gb5a9486b/rpm/el9/BUILD/ceph-18.2.0-1159-gb5a9486b/src/test/librados/snapshots_cxx.cc:463: Skipped 2023-11-02T15:38:52.857 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2023-11-02T15:38:52.857 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2023-11-02T15:38:52.857 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2023-11-02T15:38:52.857 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi174-91737-7 2023-11-02T15:38:52.857 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: waiting for snaps to purge 2023-11-02T15:38:52.858 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18073 ms) 2023-11-02T15:38:52.858 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (107679 ms total) 2023-11-02T15:38:52.858 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2023-11-02T15:38:52.858 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2023-11-02T15:38:52.858 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2023-11-02T15:38:52.859 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3381 ms) 2023-11-02T15:38:52.859 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2023-11-02T15:38:52.859 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2055 ms) 2023-11-02T15:38:52.859 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2023-11-02T15:38:52.859 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (3723 ms) 2023-11-02T15:38:52.860 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2023-11-02T15:38:52.860 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (3331 ms) 2023-11-02T15:38:52.860 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (12490 ms total) 2023-11-02T15:38:52.860 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2023-11-02T15:38:52.860 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2023-11-02T15:38:58.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN api_aio: Running main() from gmock_main.cc 2023-11-02T15:38:58.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2023-11-02T15:38:58.695 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] Global test environment set-up. 2023-11-02T15:38:58.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 26 tests from LibRadosAio 2023-11-02T15:38:58.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2023-11-02T15:38:58.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.TooBig (5353 ms) 2023-11-02T15:38:58.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2023-11-02T15:38:58.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (54540 ms) 2023-11-02T15:38:58.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2023-11-02T15:38:58.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5804 ms) 2023-11-02T15:38:58.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2023-11-02T15:38:58.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (5686 ms) 2023-11-02T15:38:58.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2023-11-02T15:38:58.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (5905 ms) 2023-11-02T15:38:58.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2023-11-02T15:38:58.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (6813 ms) 2023-11-02T15:38:58.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2023-11-02T15:38:58.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3972 ms) 2023-11-02T15:38:58.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2023-11-02T15:38:58.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (16459 ms) 2023-11-02T15:38:58.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2023-11-02T15:38:58.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (37615 ms) 2023-11-02T15:38:58.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2023-11-02T15:38:58.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (25348 ms) 2023-11-02T15:38:58.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2023-11-02T15:38:58.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (4456 ms) 2023-11-02T15:38:58.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2023-11-02T15:38:58.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (4162 ms) 2023-11-02T15:38:58.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2023-11-02T15:38:58.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (5009 ms) 2023-11-02T15:38:58.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2023-11-02T15:38:58.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3945 ms) 2023-11-02T15:38:58.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2023-11-02T15:38:58.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.Flush (3189 ms) 2023-11-02T15:38:58.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2023-11-02T15:38:58.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3255 ms) 2023-11-02T15:38:58.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2023-11-02T15:38:58.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (4975 ms) 2023-11-02T15:38:58.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2023-11-02T15:38:58.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (4713 ms) 2023-11-02T15:38:58.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2023-11-02T15:38:58.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3385 ms) 2023-11-02T15:38:58.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2023-11-02T15:38:58.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (4411 ms) 2023-11-02T15:38:58.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2023-11-02T15:38:58.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (5851 ms) 2023-11-02T15:38:58.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2023-11-02T15:38:58.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3240 ms) 2023-11-02T15:38:58.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2023-11-02T15:38:58.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (4375 ms) 2023-11-02T15:38:58.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2023-11-02T15:38:58.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2836 ms) 2023-11-02T15:38:58.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2023-11-02T15:38:58.708 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3552 ms) 2023-11-02T15:38:58.708 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2023-11-02T15:38:58.708 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3352 ms) 2023-11-02T15:38:58.708 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 26 tests from LibRadosAio (232204 ms total) 2023-11-02T15:38:58.708 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: 2023-11-02T15:38:58.709 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2023-11-02T15:39:00.858 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:13.011 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.862 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:13.015 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.911 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:13.064 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.916 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:13.069 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.929 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:13.025 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.935 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:13.031 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.953 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:13.106 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.985 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:13.081 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:00.991 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:13.144 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:08.944 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWr ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2023-11-02T15:39:08.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (5449 ms) 2023-11-02T15:39:08.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2023-11-02T15:39:08.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3514 ms) 2023-11-02T15:39:08.945 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2023-11-02T15:39:08.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4033 ms) 2023-11-02T15:39:08.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12996 ms total) 2023-11-02T15:39:08.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2023-11-02T15:39:08.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2023-11-02T15:39:08.946 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [==========] 19 tests from 4 test suites ran. (242305 ms total) 2023-11-02T15:39:08.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ PASSED ] 18 tests. 2023-11-02T15:39:08.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2023-11-02T15:39:08.947 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2023-11-02T15:39:13.226 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: Running main() from gmock_main.cc 2023-11-02T15:39:13.226 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2023-11-02T15:39:13.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] Global test environment set-up. 2023-11-02T15:39:13.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2023-11-02T15:39:13.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2023-11-02T15:39:13.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (4127 ms) 2023-11-02T15:39:13.227 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2023-11-02T15:39:13.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (79970 ms) 2023-11-02T15:39:13.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2023-11-02T15:39:13.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (13357 ms) 2023-11-02T15:39:13.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2023-11-02T15:39:13.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (14800 ms) 2023-11-02T15:39:13.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2023-11-02T15:39:13.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (35277 ms) 2023-11-02T15:39:13.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2023-11-02T15:39:13.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (19972 ms) 2023-11-02T15:39:13.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2023-11-02T15:39:13.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (4439 ms) 2023-11-02T15:39:13.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2023-11-02T15:39:13.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (4178 ms) 2023-11-02T15:39:13.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2023-11-02T15:39:13.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (4995 ms) 2023-11-02T15:39:13.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2023-11-02T15:39:13.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3954 ms) 2023-11-02T15:39:13.231 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2023-11-02T15:39:13.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3207 ms) 2023-11-02T15:39:13.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2023-11-02T15:39:13.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3227 ms) 2023-11-02T15:39:13.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2023-11-02T15:39:13.232 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3826 ms) 2023-11-02T15:39:13.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2023-11-02T15:39:13.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3281 ms) 2023-11-02T15:39:13.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2023-11-02T15:39:13.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (2585 ms) 2023-11-02T15:39:13.233 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2023-11-02T15:39:13.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3432 ms) 2023-11-02T15:39:13.234 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2023-11-02T15:39:13.235 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3180 ms) 2023-11-02T15:39:13.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2023-11-02T15:39:13.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3499 ms) 2023-11-02T15:39:13.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2023-11-02T15:39:13.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3588 ms) 2023-11-02T15:39:13.236 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3171 ms) 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3338 ms) 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3895 ms) 2023-11-02T15:39:13.237 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2023-11-02T15:39:13.238 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3543 ms) 2023-11-02T15:39:13.238 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2023-11-02T15:39:13.238 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (7485 ms) 2023-11-02T15:39:13.238 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2023-11-02T15:39:13.238 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (4014 ms) 2023-11-02T15:39:13.239 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2023-11-02T15:39:13.239 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3235 ms) 2023-11-02T15:39:13.239 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2023-11-02T15:39:14.904 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadopi_list: : cursor()=5:89d3ae78:::11:head expected=5:89d3ae78:::11:head 2023-11-02T15:39:14.904 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:89d3ae78:::11:head -> 11 2023-11-02T15:39:14.905 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=11 expected=11 2023-11-02T15:39:14.905 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:863748b0:::15:head 2023-11-02T15:39:14.905 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:863748b0:::15:head expected=5:863748b0:::15:head 2023-11-02T15:39:14.905 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:863748b0:::15:head -> 15 2023-11-02T15:39:14.905 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=15 expected=15 2023-11-02T15:39:14.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:6cac518f:::0:head 2023-11-02T15:39:14.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:6cac518f:::0:head expected=5:6cac518f:::0:head 2023-11-02T15:39:14.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:6cac518f:::0:head -> 0 2023-11-02T15:39:14.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=0 expected=0 2023-11-02T15:39:14.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:62a1935d:::14:head 2023-11-02T15:39:14.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:62a1935d:::14:head expected=5:62a1935d:::14:head 2023-11-02T15:39:14.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:62a1935d:::14:head -> 14 2023-11-02T15:39:14.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=14 expected=14 2023-11-02T15:39:14.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:5c6b0b28:::7:head 2023-11-02T15:39:14.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:5c6b0b28:::7:head expected=5:5c6b0b28:::7:head 2023-11-02T15:39:14.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:5c6b0b28:::7:head -> 7 2023-11-02T15:39:14.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=7 expected=7 2023-11-02T15:39:14.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:566253c9:::13:head 2023-11-02T15:39:14.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:566253c9:::13:head expected=5:566253c9:::13:head 2023-11-02T15:39:14.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:566253c9:::13:head -> 13 2023-11-02T15:39:14.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=13 expected=13 2023-11-02T15:39:14.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:02547ec2:::1:head 2023-11-02T15:39:14.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:02547ec2:::1:head expected=5:02547ec2:::1:head 2023-11-02T15:39:14.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:02547ec2:::1:head -> 1 2023-11-02T15:39:14.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=1 expected=1 2023-11-02T15:39:14.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 5:f905c69b:::2:head 2023-11-02T15:39:14.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=5:f905c69b:::2:head expected=5:f905c69b:::2:head 2023-11-02T15:39:14.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 5:f905c69b:::2:head -> 2 2023-11-02T15:39:14.911 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=2 expected=2 2023-11-02T15:39:14.911 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (8630 ms) 2023-11-02T15:39:14.911 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2023-11-02T15:39:14.911 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (75504 ms) 2023-11-02T15:39:14.911 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 0/5 -> MIN 5:33333333::::head 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 1/5 -> 5:33333333::::head 5:66666666::::head 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 2/5 -> 5:66666666::::head 5:99999999::::head 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 3/5 -> 5:99999999::::head 5:cccccccc::::head 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 4/5 -> 5:cccccccc::::head MAX 2023-11-02T15:39:14.912 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (43002 ms) 2023-11-02T15:39:14.913 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 7 tests from LibRadosList (234971 ms total) 2023-11-02T15:39:14.913 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2023-11-02T15:39:14.913 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 3 tests from LibRadosListEC 2023-11-02T15:39:14.913 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2023-11-02T15:39:14.913 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (2975 ms) 2023-11-02T15:39:14.914 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2023-11-02T15:39:14.914 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo2,foo3 2023-11-02T15:39:14.914 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2023-11-02T15:39:14.914 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo2 2023-11-02T15:39:14.914 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo3 2023-11-02T15:39:14.915 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo4,foo5 2023-11-02T15:39:14.915 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo4 2023-11-02T15:39:14.915 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo5 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo6,foo7 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo7 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo6 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2023-11-02T15:39:14.916 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo4 2023-11-02T15:39:14.917 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo5 2023-11-02T15:39:14.917 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo7 2023-11-02T15:39:14.917 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo6 2023-11-02T15:39:14.917 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo1 2023-11-02T15:39:14.917 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo1 2023-11-02T15:39:14.918 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo2 2023-11-02T15:39:14.918 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo3 2023-11-02T15:39:14.918 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (2013 ms) 2023-11-02T15:39:14.918 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2023-11-02T15:39:14.918 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 1 0 2023-11-02T15:39:14.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 10 0 2023-11-02T15:39:14.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 13 0 2023-11-02T15:39:20.784 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:32.938 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.849 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:33.002 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.850 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:33.003 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.896 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:32.992 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.898 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:32.995 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.913 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:33.067 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.938 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:33.092 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.944 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:33.040 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:20.966 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:33.119 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.073 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:53.227 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:53.231 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.127 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:53.221 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.127 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:06:53.276 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.130 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:53.225 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.130 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:53.282 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.133 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:53.286 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.176 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:06:53.326 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:39:41.180 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:06:53.276 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:00.971 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:13.124 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:00.992 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:13.143 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.007 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:13.160 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.020 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:13.116 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.022 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:13.118 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.043 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:13.196 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.048 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:13.201 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.049 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:13.146 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.067 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:13.220 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:01.171 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:01.162+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:02.176 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:02.173+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:03.194 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:03.186+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:04.216 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:04.214+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:04.602 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:40:04.594+0000 12a62640 -1 osd.4 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.21226.0:2 137.6 137.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e225) 2023-11-02T15:40:05.183 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:05.179+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:05.619 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:40:05.616+0000 12a62640 -1 osd.4 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.21226.0:2 137.6 137.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e225) 2023-11-02T15:40:06.219 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:06.217+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:06.593 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:40:06.589+0000 12a62640 -1 osd.4 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.21226.0:2 137.6 137.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e225) 2023-11-02T15:40:07.186 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:40:07.184+0000 12a62640 -1 osd.7 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16688.0:1 136.6s0 136.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e222) 2023-11-02T15:40:07.630 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:40:07.627+0000 12a62640 -1 osd.4 228 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.21226.0:2 137.6 137:602f83fe:::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e225) 2023-11-02T15:40:08.776 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 7 0 2023-11-02T15:40:08.777 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 14 0 2023-11-02T15:40:08.777 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 0 0 2023-11-02T15:40:08.777 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 15 0 2023-11-02T15:40:08.777 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 11 0 2023-11-02T15:40:08.777 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 5 0 2023-11-02T15:40:08.778 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 8 0 2023-11-02T15:40:08.778 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 6 0 2023-11-02T15:40:08.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 3 0 2023-11-02T15:40:08.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 4 0 2023-11-02T15:40:08.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 12 0 2023-11-02T15:40:08.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 9 0 2023-11-02T15:40:08.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2 0 2023-11-02T15:40:08.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2023-11-02T15:40:08.784 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (3689 ms) 2023-11-02T15:40:08.784 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 3 tests from LibRadosListEC (8677 ms total) 2023-11-02T15:40:08.784 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2023-11-02T15:40:08.784 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 1 test from LibRadosListNP 2023-11-02T15:40:08.784 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2023-11-02T15:40:08.785 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (49559 ms) 2023-11-02T15:40:08.785 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 1 test from LibRadosListNP (49559 ms total) 2023-11-02T15:40:08.785 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2023-11-02T15:40:08.785 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] Global test environment tear-down 2023-11-02T15:40:08.785 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [==========] 11 tests from 3 test suites ran. (302213 ms total) 2023-11-02T15:40:08.786 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ PASSED ] 11 tests. 2023-11-02T15:40:21.251 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:33.405 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.260 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:33.414 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.265 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:33.362 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.267 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:33.364 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.270 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:33.423 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.275 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:33.428 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.358 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:33.453 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.398 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:33.549 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:21.402 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:33.555 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.165 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:39.318 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.219 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:39.372 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.259 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:39.408 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.298 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:39.448 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.299 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:39.452 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.299 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:39.395 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.303 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:39.399 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.340 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:39.494 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:27.341 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:39.437 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.157 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:53.310 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.161 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:53.315 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.206 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:53.302 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.209 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:53.305 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.212 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:53.366 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.214 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:07:53.368 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.239 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:07:53.335 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.242 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:53.395 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:40:41.280 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:07:53.434 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.471 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2023-11-02T15:41:01.472 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] Global test environment set-up. 2023-11-02T15:41:01.472 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2023-11-02T15:41:01.472 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2023-11-02T15:41:01.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (1 ms) 2023-11-02T15:41:01.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (1 ms total) 2023-11-02T15:41:01.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:01.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2023-11-02T15:41:01.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: seed 91540 2023-11-02T15:41:01.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2023-11-02T15:41:01.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (297 ms) 2023-11-02T15:41:01.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2023-11-02T15:41:01.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (28064 ms) 2023-11-02T15:41:01.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2023-11-02T15:41:01.475 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (45871 ms) 2023-11-02T15:41:01.475 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2023-11-02T15:41:01.475 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (10040 ms) 2023-11-02T15:41:01.475 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2023-11-02T15:41:01.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (2817 ms) 2023-11-02T15:41:01.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2023-11-02T15:41:01.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (651 ms) 2023-11-02T15:41:01.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2023-11-02T15:41:01.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (397 ms) 2023-11-02T15:41:01.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2023-11-02T15:41:01.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (9306 ms) 2023-11-02T15:41:01.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2023-11-02T15:41:01.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (579 ms) 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (340 ms) 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (587 ms) 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2023-11-02T15:41:01.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (145 ms) 2023-11-02T15:41:01.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2023-11-02T15:41:01.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (379 ms) 2023-11-02T15:41:01.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2023-11-02T15:41:01.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: osd_max_attr_size = 0 2023-11-02T15:41:01.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2023-11-02T15:41:01.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (98875 ms) 2023-11-02T15:41:01.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2023-11-02T15:41:01.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (12472 ms) 2023-11-02T15:41:01.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2023-11-02T15:41:01.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: waiting for initial deep scrubs... 2023-11-02T15:41:01.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: done waiting, doing copies 2023-11-02T15:41:01.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: waiting for final deep scrubs... 2023-11-02T15:41:01.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: done waiting 2023-11-02T15:41:01.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (133733 ms) 2023-11-02T15:41:01.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2023-11-02T15:41:01.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (81 ms) 2023-11-02T15:41:01.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2023-11-02T15:41:01.495 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (40 ms) 2023-11-02T15:41:01.496 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2023-11-02T15:41:01.496 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5513 ms) 2023-11-02T15:41:01.496 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2023-11-02T15:41:01.496 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2023-11-02T15:41:01.505 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2023-11-02T15:41:01.505 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2023-11-02T15:41:01.505 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2023-11-02T15:41:01.506 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2023-11-02T15:41:01.536 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:13.689 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.538 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:13.692 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.573 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:13.727 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.593 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:13.746 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.596 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:13.750 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.613 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:13.709 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.615 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:13.711 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.636 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:13.789 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:01.663 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:13.757 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.639 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:33.792 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.703 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:33.856 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.706 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:33.803 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.724 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:33.819 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.766 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:33.920 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.777 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:33.930 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.794 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:33.885 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.822 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:33.970 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:21.842 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:33.996 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.708 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:58.861 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.713 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:58.867 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.760 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:58.856 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.763 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:58.859 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.763 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:58.917 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.766 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:58.920 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.776 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:08:58.927 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.833 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:08:58.986 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:46.837 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:08:58.933 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:41:59.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (350187 ms total) 2023-11-02T15:41:59.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2023-11-02T15:41:59.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2023-11-02T15:41:59.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (15140 ms) 2023-11-02T15:41:59.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (15140 ms total) 2023-11-02T15:41:59.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2023-11-02T15:41:59.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2023-11-02T15:41:59.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (554 ms) 2023-11-02T15:41:59.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2023-11-02T15:41:59.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (54 ms) 2023-11-02T15:41:59.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (608 ms total) 2023-11-02T15:41:59.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.968 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (517 ms) 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (40 ms) 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (557 ms total) 2023-11-02T15:41:59.969 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2023-11-02T15:41:59.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2023-11-02T15:41:59.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (612 ms) 2023-11-02T15:41:59.970 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2023-11-02T15:41:59.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (53 ms) 2023-11-02T15:41:59.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (665 ms total) 2023-11-02T15:41:59.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2023-11-02T15:41:59.971 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2023-11-02T15:41:59.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (16123 ms) 2023-11-02T15:41:59.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (16123 ms total) 2023-11-02T15:41:59.972 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2023-11-02T15:41:59.973 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] Global test environment tear-down 2023-11-02T15:41:59.973 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (413393 ms total) 2023-11-02T15:41:59.973 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ PASSED ] 31 tests. 2023-11-02T15:42:06.787 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:18.939 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.788 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:18.941 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.823 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:18.976 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.836 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:18.932 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.837 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:18.991 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.838 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:18.934 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.845 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:18.992 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.873 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:18.967 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:06.877 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:19.031 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:07.963 INFO:tasks.workunit.client.0.smithi174.stdout:ite 2023-11-02T15:42:07.963 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7128 ms) 2023-11-02T15:42:07.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2023-11-02T15:42:07.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7405 ms) 2023-11-02T15:42:07.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2023-11-02T15:42:07.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7063 ms) 2023-11-02T15:42:07.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2023-11-02T15:42:07.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6968 ms) 2023-11-02T15:42:07.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2023-11-02T15:42:07.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (43962 ms) 2023-11-02T15:42:07.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2023-11-02T15:42:07.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7653 ms) 2023-11-02T15:42:07.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2023-11-02T15:42:07.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (8453 ms) 2023-11-02T15:42:07.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2023-11-02T15:42:07.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7525 ms) 2023-11-02T15:42:07.967 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2023-11-02T15:42:07.974 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7047 ms) 2023-11-02T15:42:07.974 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2023-11-02T15:42:07.974 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (27262 ms) 2023-11-02T15:42:07.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2023-11-02T15:42:07.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6775 ms) 2023-11-02T15:42:07.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2023-11-02T15:42:07.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6738 ms) 2023-11-02T15:42:07.975 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2023-11-02T15:42:07.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7755 ms) 2023-11-02T15:42:07.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2023-11-02T15:42:07.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7411 ms) 2023-11-02T15:42:07.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2023-11-02T15:42:07.976 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (22106 ms) 2023-11-02T15:42:07.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2023-11-02T15:42:07.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (8015 ms) 2023-11-02T15:42:07.977 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (189266 ms total) 2023-11-02T15:42:07.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: 2023-11-02T15:42:07.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] Global test environment tear-down 2023-11-02T15:42:07.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (421470 ms total) 2023-11-02T15:42:07.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ PASSED ] 42 tests. 2023-11-02T15:42:21.212 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:21.210+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:22.180 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:22.177+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:23.147 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:23.144+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:24.135 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:24.128+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:25.124 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:25.122+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:26.114 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:26.111+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:26.680 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:38.833 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.683 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:38.837 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.702 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:38.854 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.729 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:38.825 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.730 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:38.884 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.732 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:38.829 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.732 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:38.885 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.751 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:38.904 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:26.752 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:38.848 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:27.140 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:27.137+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:28.157 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:28.155+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:29.131 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:29.129+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:30.138 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:30.135+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:31.177 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:31.173+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:31.671 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:43.824 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.679 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:43.833 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.697 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:43.850 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.732 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:43.828 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.738 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:43.830 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.749 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:43.845 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.749 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:43.903 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.756 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:43.908 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:31.772 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:43.925 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:32.164 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:32.161+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:33.144 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:33.140+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:34.112 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:34.109+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:35.159 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:35.156+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:36.122 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:36.119+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:37.082 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:37.080+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:38.059 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:38.056+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:39.065 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:39.063+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:40.020 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:40.018+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:41.003 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:41.001+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:42.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:42.052+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:43.028 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:43.026+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:44.016 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:44.013+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:45.026 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:45.023+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:46.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:46.052+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:46.776 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:58.929 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.781 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:58.933 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.798 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:58.951 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.831 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:58.927 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.833 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:58.930 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.835 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:58.988 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.837 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:58.990 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.850 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:09:58.946 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:46.854 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:59.007 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:42:47.038 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:47.035+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:48.033 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:48.030+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:49.074 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:49.072+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:50.121 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:50.118+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:51.097 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:51.095+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:52.059 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:52.057+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:53.018 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:53.013+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:54.006 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:54.001+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:54.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:42:54.296+0000 12a62640 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:30.034630+0000 front 2023-11-02T15:41:30.038415+0000 (oldest deadline 2023-11-02T15:42:54.130864+0000) 2023-11-02T15:42:54.479 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:42:54.473+0000 12a62640 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.693495+0000 front 2023-11-02T15:41:32.697668+0000 (oldest deadline 2023-11-02T15:42:54.377513+0000) 2023-11-02T15:42:54.990 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:54.988+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:55.166 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:42:55.155+0000 12a62640 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.869318+0000 front 2023-11-02T15:41:31.931881+0000 (oldest deadline 2023-11-02T15:42:54.737469+0000) 2023-11-02T15:42:55.306 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:42:55.301+0000 12a62640 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:30.034630+0000 front 2023-11-02T15:41:30.038415+0000 (oldest deadline 2023-11-02T15:42:54.130864+0000) 2023-11-02T15:42:55.446 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:42:55.440+0000 12a62640 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.693495+0000 front 2023-11-02T15:41:32.697668+0000 (oldest deadline 2023-11-02T15:42:54.377513+0000) 2023-11-02T15:42:55.618 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:42:55.607+0000 12a62640 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.741134+0000 front 2023-11-02T15:41:31.689868+0000 (oldest deadline 2023-11-02T15:42:55.162176+0000) 2023-11-02T15:42:56.009 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:56.005+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:56.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:42:56.166+0000 12a62640 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.869318+0000 front 2023-11-02T15:41:31.931881+0000 (oldest deadline 2023-11-02T15:42:54.737469+0000) 2023-11-02T15:42:56.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:42:56.312+0000 12a62640 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:30.034630+0000 front 2023-11-02T15:41:30.038415+0000 (oldest deadline 2023-11-02T15:42:54.130864+0000) 2023-11-02T15:42:56.476 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:42:56.474+0000 12a62640 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.693495+0000 front 2023-11-02T15:41:32.697668+0000 (oldest deadline 2023-11-02T15:42:54.377513+0000) 2023-11-02T15:42:56.630 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:42:56.627+0000 12a62640 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.741134+0000 front 2023-11-02T15:41:31.689868+0000 (oldest deadline 2023-11-02T15:42:55.162176+0000) 2023-11-02T15:42:56.968 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:56.962+0000 12a62640 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.940097+0000 front 2023-11-02T15:41:31.935131+0000 (oldest deadline 2023-11-02T15:42:56.032777+0000) 2023-11-02T15:42:56.971 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:56.968+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:57.158 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:42:57.155+0000 12a62640 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.869318+0000 front 2023-11-02T15:41:31.931881+0000 (oldest deadline 2023-11-02T15:42:54.737469+0000) 2023-11-02T15:42:57.353 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:42:57.351+0000 12a62640 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:30.034630+0000 front 2023-11-02T15:41:30.038415+0000 (oldest deadline 2023-11-02T15:42:54.130864+0000) 2023-11-02T15:42:57.479 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:42:57.472+0000 12a62640 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.693495+0000 front 2023-11-02T15:41:32.697668+0000 (oldest deadline 2023-11-02T15:42:54.377513+0000) 2023-11-02T15:42:57.594 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:42:57.587+0000 12a62640 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.741134+0000 front 2023-11-02T15:41:31.689868+0000 (oldest deadline 2023-11-02T15:42:55.162176+0000) 2023-11-02T15:42:57.734 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:42:57.728+0000 12a62640 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.978771+0000 front 2023-11-02T15:41:33.081791+0000 (oldest deadline 2023-11-02T15:42:56.894601+0000) 2023-11-02T15:42:57.973 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:57.968+0000 12a62640 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.940097+0000 front 2023-11-02T15:41:31.935131+0000 (oldest deadline 2023-11-02T15:42:56.032777+0000) 2023-11-02T15:42:57.973 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:57.969+0000 12a62640 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:58.017 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:42:58.010+0000 12a62640 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.440829+0000 front 2023-11-02T15:41:31.441617+0000 (oldest deadline 2023-11-02T15:42:57.332486+0000) 2023-11-02T15:42:58.185 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:42:58.183+0000 12a62640 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.869318+0000 front 2023-11-02T15:41:31.931881+0000 (oldest deadline 2023-11-02T15:42:54.737469+0000) 2023-11-02T15:42:58.555 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:42:58.551+0000 12a62640 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.741134+0000 front 2023-11-02T15:41:31.689868+0000 (oldest deadline 2023-11-02T15:42:55.162176+0000) 2023-11-02T15:42:58.704 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:42:58.700+0000 12a62640 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:32.978771+0000 front 2023-11-02T15:41:33.081791+0000 (oldest deadline 2023-11-02T15:42:56.894601+0000) 2023-11-02T15:42:58.933 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:42:58.932+0000 12a62640 -1 osd.7 333 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15648.0:1 168.3s0 168.c2db6a63 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e320) 2023-11-02T15:42:59.214 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:42:59.210+0000 12a62640 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.175:6828 osd.6 since back 2023-11-02T15:41:31.869318+0000 front 2023-11-02T15:41:31.931881+0000 (oldest deadline 2023-11-02T15:42:54.737469+0000) 2023-11-02T15:43:06.877 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:19.030 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.920 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:19.074 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.931 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:19.027 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.931 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:19.085 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.933 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:19.087 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.948 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:19.040 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.973 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:19.127 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:06.974 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:19.070 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:07.016 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:19.169 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.859 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:39.012 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:39.004 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.909 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:39.062 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.910 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:39.007 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.950 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:39.046 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.955 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:39.108 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.956 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:39.110 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.978 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:39.131 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:26.999 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:39.152 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.180 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:59.333 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.182 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:59.335 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.226 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:10:59.378 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.233 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:59.330 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.234 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:59.331 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.243 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:59.394 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.246 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:59.399 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.271 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:10:59.368 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:47.285 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:10:59.438 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:43:53.840 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:43:53.837+0000 14265640 -1 osd.6 385 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:44:07.249 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:19.399 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.249 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:19.401 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.295 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:19.449 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.305 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:19.458 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.310 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:19.460 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.314 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:19.410 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.315 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:19.412 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.348 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:19.502 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:07.357 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:19.453 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:12.185 INFO:tasks.workunit.client.0.smithi174.stdout:sAio.RoundTripAppendPP (3157 ms) 2023-11-02T15:44:12.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2023-11-02T15:44:12.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3673 ms) 2023-11-02T15:44:12.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2023-11-02T15:44:12.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3385 ms) 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3416 ms) 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: setting pool EIO 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: max_success 71, min_failed 72 2023-11-02T15:44:12.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (7899 ms) 2023-11-02T15:44:12.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2023-11-02T15:44:12.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (37002 ms) 2023-11-02T15:44:12.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (302108 ms total) 2023-11-02T15:44:12.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2023-11-02T15:44:12.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2023-11-02T15:44:12.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2023-11-02T15:44:12.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (11362 ms) 2023-11-02T15:44:12.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2023-11-02T15:44:12.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (10512 ms) 2023-11-02T15:44:12.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2023-11-02T15:44:12.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (46333 ms) 2023-11-02T15:44:12.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2023-11-02T15:44:12.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3120 ms) 2023-11-02T15:44:12.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (71327 ms total) 2023-11-02T15:44:12.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2023-11-02T15:44:12.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2023-11-02T15:44:12.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2023-11-02T15:44:12.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3860 ms) 2023-11-02T15:44:12.191 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3860 ms total) 2023-11-02T15:44:12.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2023-11-02T15:44:12.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2023-11-02T15:44:12.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2023-11-02T15:44:12.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (16026 ms) 2023-11-02T15:44:12.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2023-11-02T15:44:12.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7826 ms) 2023-11-02T15:44:12.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2023-11-02T15:44:12.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (74990 ms) 2023-11-02T15:44:12.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2023-11-02T15:44:12.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7610 ms) 2023-11-02T15:44:12.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2023-11-02T15:44:12.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2775 ms) 2023-11-02T15:44:12.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2023-11-02T15:44:12.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7152 ms) 2023-11-02T15:44:12.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2023-11-02T15:44:12.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7406 ms) 2023-11-02T15:44:12.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2023-11-02T15:44:12.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7927 ms) 2023-11-02T15:44:12.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2023-11-02T15:44:12.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (6806 ms) 2023-11-02T15:44:12.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2023-11-02T15:44:12.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (8109 ms) 2023-11-02T15:44:12.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2023-11-02T15:44:12.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7805 ms) 2023-11-02T15:44:12.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2023-11-02T15:44:12.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7404 ms) 2023-11-02T15:44:12.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2023-11-02T15:44:12.197 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (6560 ms) 2023-11-02T15:44:27.335 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:39.489 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.340 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:39.494 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.386 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:39.482 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.387 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:39.484 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.420 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:39.572 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.421 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:39.573 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.425 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:39.578 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.429 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:39.526 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:27.482 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:39.635 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.865 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:41.018 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.871 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:41.025 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.886 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:41.040 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.917 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:41.014 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.922 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:41.018 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.923 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:41.076 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.924 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:41.078 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.941 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:41.037 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:28.942 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:41.096 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.413 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:59.566 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.416 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:59.570 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.458 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:59.612 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.479 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:59.575 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.484 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:59.637 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.485 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:59.638 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.487 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:59.584 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.532 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:11:59.628 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:47.539 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:59.693 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.418 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:09.571 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.421 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:09.574 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.456 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:09.610 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.467 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:09.564 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.473 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:09.569 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.475 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:09.628 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.478 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:09.632 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.499 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:09.595 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:44:57.505 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:09.659 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.480 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:19.633 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.485 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:19.637 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.499 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:19.653 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.529 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:19.625 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.530 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:19.626 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.531 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:19.685 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.533 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:19.686 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.546 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:19.642 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:07.547 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:19.701 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.442 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:29.596 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.444 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:29.598 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.459 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:29.612 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.489 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:29.586 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.491 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:29.587 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.491 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:29.644 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.492 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:29.646 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.505 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:29.602 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:17.513 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:29.666 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:25.138 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:25.131+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:26.166 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:26.164+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:27.179 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:27.177+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:27.474 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:39.628 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.478 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:39.632 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.493 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:39.646 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.521 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:39.617 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.522 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:39.675 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.523 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:39.620 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.526 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:39.680 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.536 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:39.633 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:27.540 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:39.693 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:28.174 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:28.171+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:29.128 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:29.126+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:30.087 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:30.085+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:31.073 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:31.070+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:32.088 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:32.086+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:32.440 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:44.594 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.490 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:44.642 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.491 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:44.587 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.494 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:44.590 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.503 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:44.657 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.507 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:44.603 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.535 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:44.689 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.538 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:44.692 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:32.550 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:44.703 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:33.121 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:33.118+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:34.172 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:34.169+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:35.207 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:35.205+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:36.218 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:36.216+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:37.219 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:37.217+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:38.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:38.188+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:39.164 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:39.161+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:40.123 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:40.120+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:41.129 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:41.127+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:42.086 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:42.083+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:43.083 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:43.080+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:44.102 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:44.099+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:45.102 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:45.099+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:46.113 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:46.109+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:47.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:47.080+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:47.574 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:59.727 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.577 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:59.731 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.604 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:12:59.757 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.624 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:59.721 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.626 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:59.723 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.631 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:59.785 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.636 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:59.790 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.647 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:12:59.740 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:47.655 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:12:59.808 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.074 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:48.068+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:48.217 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:00.314 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.218 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:00.315 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.226 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:00.323 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.240 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:00.394 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.241 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:00.395 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.254 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:00.407 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.255 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:00.408 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.257 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:00.410 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:48.266 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:00.419 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:45:49.065 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:49.061+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:50.068 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:50.066+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:51.114 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:51.112+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:52.115 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:52.113+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:53.108 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:53.106+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:54.107 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:54.104+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:55.156 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:55.154+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:56.200 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:56.198+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:57.249 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:57.247+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:58.214 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:58.211+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:45:59.172 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:45:59.169+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:00.133 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:00.131+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:01.136 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:01.134+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:02.163 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:02.161+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:02.610 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:14.763 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.615 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:14.768 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.615 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:14.712 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.619 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:14.715 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.635 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:14.788 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.637 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:14.733 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.662 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:14.816 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.667 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:14.821 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:02.693 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:14.847 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:03.132 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:03.129+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:04.140 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:04.137+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:05.116 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:05.114+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:05.496 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:05.482+0000 15267640 -1 bdev(0x18e80120 /var/lib/ceph/osd/ceph-2/block) debug_aio_unlink stalled aio 0x18eed2e0 since 2023-11-02T15:45:12.015308+0000, timeout is 5s 2023-11-02T15:46:05.496 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:05.483+0000 16269640 -1 bdev(0x193b93d0 /var/lib/ceph/osd/ceph-2/block) debug_aio_unlink stalled aio 0x1916a5b0 since 2023-11-02T15:45:12.015761+0000, timeout is 5s 2023-11-02T15:46:06.092 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:06.090+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:07.104 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:07.102+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:07.704 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:19.857 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.707 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:19.861 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.718 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:19.872 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.752 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:19.849 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.753 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:19.850 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.754 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:19.907 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.755 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:19.909 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.764 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:19.860 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:07.766 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:19.920 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:08.067 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:08.065+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:09.037 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:09.035+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:10.025 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:10.022+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:11.037 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:11.035+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:12.000 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:11.998+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:12.985 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:46:12.972+0000 12a62640 -1 osd.6 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:49.046334+0000 front 2023-11-02T15:44:49.047816+0000 (oldest deadline 2023-11-02T15:46:12.543796+0000) 2023-11-02T15:46:13.046 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:13.043+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:13.105 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:46:13.103+0000 12a62640 -1 osd.5 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:46.682650+0000 front 2023-11-02T15:44:46.662661+0000 (oldest deadline 2023-11-02T15:46:12.632365+0000) 2023-11-02T15:46:13.431 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:46:13.429+0000 12a62640 -1 osd.1 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:48.578261+0000 front 2023-11-02T15:44:48.579230+0000 (oldest deadline 2023-11-02T15:46:13.274610+0000) 2023-11-02T15:46:13.707 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:46:13.704+0000 12a62640 -1 osd.7 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:50.294089+0000 front 2023-11-02T15:44:50.300153+0000 (oldest deadline 2023-11-02T15:46:13.191863+0000) 2023-11-02T15:46:14.002 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:46:13.999+0000 12a62640 -1 osd.6 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:49.046334+0000 front 2023-11-02T15:44:49.047816+0000 (oldest deadline 2023-11-02T15:46:12.543796+0000) 2023-11-02T15:46:14.074 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:14.071+0000 12a62640 -1 osd.3 452 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T15:44:48.306138+0000 front 2023-11-02T15:44:48.303324+0000 (oldest deadline 2023-11-02T15:46:13.615125+0000) 2023-11-02T15:46:14.075 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:14.072+0000 12a62640 -1 osd.3 452 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:15.045 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:46:15.042+0000 12a62640 -1 osd.3 453 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.16868.0:1 192.7s0 192.2927bfdf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446) 2023-11-02T15:46:15.793 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:15.790+0000 14265640 -1 osd.2 454 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:46:17.616 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:29.770 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.674 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:29.828 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.698 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:29.852 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.723 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:29.819 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.727 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:29.824 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.731 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:29.884 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.733 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:29.886 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.750 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:29.846 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:17.753 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:29.907 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:18.525 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2023-11-02T15:46:18.525 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3396 ms) 2023-11-02T15:46:18.526 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2023-11-02T15:46:18.526 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (6669 ms) 2023-11-02T15:46:18.526 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2023-11-02T15:46:18.526 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7274 ms) 2023-11-02T15:46:18.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2023-11-02T15:46:18.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7515 ms) 2023-11-02T15:46:18.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2023-11-02T15:46:18.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (6951 ms) 2023-11-02T15:46:18.527 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2023-11-02T15:46:18.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7470 ms) 2023-11-02T15:46:18.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2023-11-02T15:46:18.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (87063 ms) 2023-11-02T15:46:18.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (294735 ms total) 2023-11-02T15:46:18.528 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2023-11-02T15:46:18.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] Global test environment tear-down 2023-11-02T15:46:18.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (672030 ms total) 2023-11-02T15:46:18.529 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ PASSED ] 57 tests. 2023-11-02T15:46:27.690 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:39.844 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.694 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:39.848 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.712 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:39.866 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.740 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:39.894 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.745 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:39.899 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.746 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:39.843 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.747 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:39.844 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.761 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:39.914 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:27.763 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:39.860 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.662 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:44.815 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.711 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:44.864 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.726 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:44.880 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.759 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:44.855 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.759 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:44.856 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.761 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:44.914 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.764 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:44.918 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.771 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:44.868 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:32.777 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:44.931 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.744 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:59.897 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.746 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:59.900 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.762 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:13:59.916 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.794 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:59.947 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.795 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:59.891 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.798 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:59.952 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.798 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:59.895 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.812 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:13:59.909 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:47.813 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:13:59.967 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:49.099 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:49.093+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:50.112 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:50.110+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:51.148 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:51.146+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:52.106 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:52.104+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:52.762 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:04.915 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.765 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:04.918 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.774 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:04.927 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.809 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:04.906 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.810 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:04.963 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.812 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:04.909 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.813 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:04.966 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.823 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:04.920 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:52.825 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:04.978 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:46:53.139 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:53.137+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:54.157 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:54.154+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:55.111 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:55.108+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:56.134 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:56.132+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:57.173 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:57.171+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:58.146 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:58.144+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:46:59.140 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:46:59.139+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:00.115 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:00.113+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:01.148 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:01.145+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:02.156 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:02.154+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:03.145 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:03.143+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:04.110 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:04.107+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:05.150 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:05.147+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:06.122 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:06.120+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:07.104 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:07.102+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:08.079 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:08.077+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:09.058 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:09.055+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:10.061 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:10.058+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:11.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:11.019+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:12.043 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:12.041+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:12.769 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:24.922 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.815 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:24.969 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.818 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:24.914 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.821 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:24.917 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.845 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:24.941 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.845 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:24.999 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.867 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:25.021 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.870 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:25.024 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:12.894 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:25.048 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:13.067 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:13.064+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:14.019 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:14.017+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:15.004 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:15.001+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:16.033 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:16.030+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:16.994 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:16.991+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:17.830 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:29.983 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:29.985 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.842 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:29.996 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.877 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:30.030 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.879 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:30.033 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.881 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:29.977 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.882 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:29.978 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.892 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:29.989 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.892 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:30.045 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:17.999 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:17.997+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:18.979 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:18.976+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:19.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:19.987+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:21.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:21.020+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:21.988 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:21.985+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:23.017 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:23.015+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:24.012 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:24.010+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:25.029 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:25.027+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:26.038 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:47:26.035+0000 12a62640 -1 osd.2 458 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:2083 191.19 191.19 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e457) 2023-11-02T15:47:33.037 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:45.190 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.042 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:45.196 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.062 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:14:45.216 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.095 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:45.248 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.096 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:45.250 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.102 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:45.198 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.102 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:45.199 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.117 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:14:45.270 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:33.123 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:14:45.219 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.937 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:00.091 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.940 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:00.094 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.971 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:00.124 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.990 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:00.087 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.992 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:00.146 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.992 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:00.089 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:47.995 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:00.148 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:48.022 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:00.118 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:48.024 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:00.178 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:52.958 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:05.111 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:52.960 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:05.113 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:52.972 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:05.126 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.007 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:05.161 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.008 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:05.104 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.010 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:05.164 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.010 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:05.107 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.021 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:05.174 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:47:53.021 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:05.118 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:07.991 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:20.144 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:07.996 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:20.150 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.018 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:20.172 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.050 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:20.203 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.054 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:20.208 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.060 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:20.156 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.062 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:20.159 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.070 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:20.224 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:08.079 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:20.175 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.004 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:25.157 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.008 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:25.162 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.025 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:25.178 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.057 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:25.211 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.059 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:25.155 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.060 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:25.214 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.062 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:25.159 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.077 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:25.231 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:13.078 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:25.175 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.122 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:35.276 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.176 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:35.272 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.176 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:35.273 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.186 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:35.340 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.201 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:35.298 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.212 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:35.366 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.271 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:35.424 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.272 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:35.425 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:23.294 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:35.448 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.163 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:40.317 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.169 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:40.322 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.189 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:40.342 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.219 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:40.316 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.223 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:40.320 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.226 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:40.380 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.229 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:40.383 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.241 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:40.338 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:28.249 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:40.402 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.339 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:45.492 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.341 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:45.495 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.360 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:45.514 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.402 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:45.499 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.404 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:45.501 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:45.562 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.410 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:45.563 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.419 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:45.516 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:33.428 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:45.581 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.176 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:50.330 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.221 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:50.318 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.222 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:50.319 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.225 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:50.379 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.226 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:50.379 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.229 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:50.383 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.238 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:15:50.335 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.244 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:15:50.398 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:38.248 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:15:50.402 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.273 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:05.426 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.277 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:05.431 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.310 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:05.463 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.329 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:05.483 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.331 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:05.485 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.338 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:05.435 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.342 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:05.439 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.373 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:05.527 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.374 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:05.470 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.960 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:06.057 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.961 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:06.058 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:53.969 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:06.066 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:06.244 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.092 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:06.245 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.101 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:06.255 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.109 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:06.262 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.110 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:06.264 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:48:54.120 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:06.274 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.076 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:16.230 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.085 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:16.239 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.091 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:16.187 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.094 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:16.190 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.098 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:16.251 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.101 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:16.255 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.174 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:16.327 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.184 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:16.338 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:04.201 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:16.298 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.459 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:20.613 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.464 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:20.618 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.486 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:20.639 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.522 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:20.619 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.526 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:20.623 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.535 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:20.688 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.536 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:20.690 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.539 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:20.635 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:08.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:20.710 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:12.719 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2023-11-02T15:49:12.719 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] Global test environment set-up. 2023-11-02T15:49:12.720 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2023-11-02T15:49:12.720 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: seed 91559 2023-11-02T15:49:12.720 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2023-11-02T15:49:12.720 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (25373 ms) 2023-11-02T15:49:12.720 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2023-11-02T15:49:12.721 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (40375 ms) 2023-11-02T15:49:12.721 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2023-11-02T15:49:12.721 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (169 ms) 2023-11-02T15:49:12.721 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (65918 ms total) 2023-11-02T15:49:12.722 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2023-11-02T15:49:12.722 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2023-11-02T15:49:12.722 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2023-11-02T15:49:12.722 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (38249 ms) 2023-11-02T15:49:12.723 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2023-11-02T15:49:12.723 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (63875 ms) 2023-11-02T15:49:12.723 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2023-11-02T15:49:12.723 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (20366 ms) 2023-11-02T15:49:12.723 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2023-11-02T15:49:12.724 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [4] 2023-11-02T15:49:12.724 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [5,4] 2023-11-02T15:49:12.724 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [6,5,4] 2023-11-02T15:49:12.724 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [7,6,5,4] 2023-11-02T15:49:12.724 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting some heads 2023-11-02T15:49:12.725 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 7 2023-11-02T15:49:12.725 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 6 2023-11-02T15:49:12.725 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 5 2023-11-02T15:49:12.725 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 4 2023-11-02T15:49:12.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrubs... 2023-11-02T15:49:12.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2023-11-02T15:49:12.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (340600 ms) 2023-11-02T15:49:12.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2023-11-02T15:49:12.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9788 ms) 2023-11-02T15:49:12.727 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2023-11-02T15:49:12.727 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8879 ms) 2023-11-02T15:49:12.727 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2023-11-02T15:49:12.727 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8233 ms) 2023-11-02T15:49:12.728 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2023-11-02T15:49:12.728 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (10097 ms) 2023-11-02T15:49:12.728 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2023-11-02T15:49:12.728 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10882 ms) 2023-11-02T15:49:12.728 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2023-11-02T15:49:12.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (156609 ms) 2023-11-02T15:49:12.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2023-11-02T15:49:12.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10920 ms) 2023-11-02T15:49:12.729 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2023-11-02T15:49:12.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (38430 ms) 2023-11-02T15:49:12.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2023-11-02T15:49:12.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (10051 ms) 2023-11-02T15:49:12.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2023-11-02T15:49:12.730 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (11136 ms) 2023-11-02T15:49:12.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2023-11-02T15:49:12.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (14385 ms) 2023-11-02T15:49:12.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2023-11-02T15:49:12.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8934 ms) 2023-11-02T15:49:12.731 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2023-11-02T15:49:12.732 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (10797 ms) 2023-11-02T15:49:13.498 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:25.651 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.500 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:25.654 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.518 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:25.672 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.559 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:25.655 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.568 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:25.665 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.578 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:25.732 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.585 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:25.738 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.592 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:25.689 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:13.605 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:25.758 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.419 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:35.572 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.477 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:35.573 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.478 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:35.575 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.488 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:35.642 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.509 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:35.604 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.539 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:35.692 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:35.709 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.571 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:35.725 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:23.603 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:35.757 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.572 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:45.726 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.575 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:45.729 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.594 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:45.747 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.625 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:45.721 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.626 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:45.722 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.634 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:45.788 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.635 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:45.789 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.644 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:45.741 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:33.656 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:45.810 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:34.530 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2023-11-02T15:49:34.530 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: hmm, no HitSet yet 2023-11-02T15:49:34.530 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: ok, hit_set contains 200:602f83fe:::foo:head 2023-11-02T15:49:34.531 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9421 ms) 2023-11-02T15:49:34.531 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2023-11-02T15:49:34.531 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg_num = 32 2023-11-02T15:49:34.531 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 0 ls 1698940169,0 2023-11-02T15:49:34.531 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 1 ls 1698940169,0 2023-11-02T15:49:34.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 2 ls 1698940170,0 2023-11-02T15:49:34.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 3 ls 1698940170,0 2023-11-02T15:49:34.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 4 ls 1698940169,0 2023-11-02T15:49:34.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 5 ls 1698940170,0 2023-11-02T15:49:34.532 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 6 ls 1698940170,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 7 ls 1698940170,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 8 ls 1698940169,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 9 ls 1698940170,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 10 ls 1698940169,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 11 ls 1698940169,0 2023-11-02T15:49:34.533 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 12 ls 1698940170,0 2023-11-02T15:49:34.534 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 13 ls 1698940170,0 2023-11-02T15:49:34.534 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 14 ls 1698940170,0 2023-11-02T15:49:34.534 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 15 ls 1698940170,0 2023-11-02T15:49:34.534 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 16 ls 1698940170,0 2023-11-02T15:49:34.534 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 17 ls 1698940170,0 2023-11-02T15:49:34.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 18 ls 1698940170,0 2023-11-02T15:49:34.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 19 ls 1698940170,0 2023-11-02T15:49:34.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 20 ls 1698940170,0 2023-11-02T15:49:34.535 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 21 ls 1698940170,0 2023-11-02T15:49:34.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 22 ls 1698940170,0 2023-11-02T15:49:34.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 23 ls 1698940170,0 2023-11-02T15:49:34.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 24 ls 1698940170,0 2023-11-02T15:49:34.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 25 ls 1698940170,0 2023-11-02T15:49:34.536 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 26 ls 1698940170,0 2023-11-02T15:49:34.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 27 ls 1698940170,0 2023-11-02T15:49:34.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 28 ls 1698940170,0 2023-11-02T15:49:34.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 29 ls 1698940170,0 2023-11-02T15:49:34.537 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 30 ls 1698940170,0 2023-11-02T15:49:34.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 31 ls 1698940170,0 2023-11-02T15:49:34.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg_num = 32 2023-11-02T15:49:34.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6cac518f:::0:head 2023-11-02T15:49:34.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:02547ec2:::1:head 2023-11-02T15:49:34.538 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f905c69b:::2:head 2023-11-02T15:49:34.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:cfc208b3:::3:head 2023-11-02T15:49:34.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d83876eb:::4:head 2023-11-02T15:49:34.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b29083e3:::5:head 2023-11-02T15:49:34.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c4fdafeb:::6:head 2023-11-02T15:49:34.539 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:5c6b0b28:::7:head 2023-11-02T15:49:34.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:bd63b0f1:::8:head 2023-11-02T15:49:34.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e960b815:::9:head 2023-11-02T15:49:34.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:52ea6a34:::10:head 2023-11-02T15:49:34.540 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:89d3ae78:::11:head 2023-11-02T15:49:34.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:de5d7c5f:::12:head 2023-11-02T15:49:34.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:566253c9:::13:head 2023-11-02T15:49:34.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:62a1935d:::14:head 2023-11-02T15:49:34.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:863748b0:::15:head 2023-11-02T15:49:34.541 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3958e169:::16:head 2023-11-02T15:49:34.542 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4d4dabf9:::17:head 2023-11-02T15:49:34.542 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:8391935d:::18:head 2023-11-02T15:49:34.542 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:28883081:::19:head 2023-11-02T15:49:34.542 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:69259c59:::20:head 2023-11-02T15:49:34.543 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4bdb80b7:::21:head 2023-11-02T15:49:34.543 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a11c5d71:::22:head 2023-11-02T15:49:34.543 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:271af37b:::23:head 2023-11-02T15:49:34.543 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:95b121be:::24:head 2023-11-02T15:49:34.543 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:58d1031b:::25:head 2023-11-02T15:49:34.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:0a050783:::26:head 2023-11-02T15:49:34.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c709704c:::27:head 2023-11-02T15:49:34.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:cbe56eaf:::28:head 2023-11-02T15:49:34.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:86b4b162:::29:head 2023-11-02T15:49:34.544 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:70d89383:::30:head 2023-11-02T15:49:34.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:dd450c7c:::31:head 2023-11-02T15:49:34.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6d5729b1:::32:head 2023-11-02T15:49:34.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c388f3fb:::33:head 2023-11-02T15:49:34.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:56cfea31:::34:head 2023-11-02T15:49:34.545 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9dbc1bf7:::35:head 2023-11-02T15:49:34.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:40b74ccd:::36:head 2023-11-02T15:49:34.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4d5aaf42:::37:head 2023-11-02T15:49:34.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:920f362c:::38:head 2023-11-02T15:49:34.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6cc53222:::39:head 2023-11-02T15:49:34.546 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9cad833f:::40:head 2023-11-02T15:49:34.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:1ea84d41:::41:head 2023-11-02T15:49:34.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c4480ef6:::42:head 2023-11-02T15:49:34.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a694361e:::43:head 2023-11-02T15:49:34.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d1bd33e9:::44:head 2023-11-02T15:49:34.547 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ddc2cd5d:::45:head 2023-11-02T15:49:34.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:2b782207:::46:head 2023-11-02T15:49:34.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7b187fca:::47:head 2023-11-02T15:49:34.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:90ecdf6f:::48:head 2023-11-02T15:49:34.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a5ed95fe:::49:head 2023-11-02T15:49:34.548 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ea0eaa55:::50:head 2023-11-02T15:49:34.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f33ef17b:::51:head 2023-11-02T15:49:34.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a0d1b2f6:::52:head 2023-11-02T15:49:34.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:60c5229e:::53:head 2023-11-02T15:49:34.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:edcbc575:::54:head 2023-11-02T15:49:34.549 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:102cf253:::55:head 2023-11-02T15:49:34.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:efb7fb0b:::56:head 2023-11-02T15:49:34.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:50d0a326:::57:head 2023-11-02T15:49:34.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d4dc5daf:::58:head 2023-11-02T15:49:34.550 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3a130462:::59:head 2023-11-02T15:49:34.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ec87ed71:::60:head 2023-11-02T15:49:34.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d5bc9454:::61:head 2023-11-02T15:49:34.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3ddfe313:::62:head 2023-11-02T15:49:34.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7c2816b9:::63:head 2023-11-02T15:49:34.551 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:47e00e4d:::64:head 2023-11-02T15:49:34.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c6410c18:::65:head 2023-11-02T15:49:34.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b48ed237:::66:head 2023-11-02T15:49:34.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:cd63ad31:::67:head 2023-11-02T15:49:34.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b179e92b:::68:head 2023-11-02T15:49:34.552 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:0d9f741a:::69:head 2023-11-02T15:49:34.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6d3352ae:::70:head 2023-11-02T15:49:34.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c6d5c19e:::71:head 2023-11-02T15:49:34.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:bc4729c3:::72:head 2023-11-02T15:49:34.553 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:77e930b9:::73:head 2023-11-02T15:49:34.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:0abeecfd:::74:head 2023-11-02T15:49:34.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b7c37e15:::75:head 2023-11-02T15:49:34.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b6378398:::76:head 2023-11-02T15:49:34.554 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:02bd68de:::77:head 2023-11-02T15:49:34.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:cc795d2d:::78:head 2023-11-02T15:49:34.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:630d4fea:::79:head 2023-11-02T15:49:34.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e0d29ef5:::80:head 2023-11-02T15:49:34.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:fd6f13d2:::81:head 2023-11-02T15:49:34.555 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:606461d5:::82:head 2023-11-02T15:49:34.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:eadbdc43:::83:head 2023-11-02T15:49:34.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:8761d0bb:::84:head 2023-11-02T15:49:34.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9ef0186f:::85:head 2023-11-02T15:49:34.556 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e0d41294:::86:head 2023-11-02T15:49:34.557 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:961de695:::87:head 2023-11-02T15:49:34.557 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:1423148f:::88:head 2023-11-02T15:49:34.557 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:633a8fa2:::89:head 2023-11-02T15:49:34.557 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a8653809:::90:head 2023-11-02T15:49:34.557 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3dac8b33:::91:head 2023-11-02T15:49:34.558 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:35aad435:::92:head 2023-11-02T15:49:34.558 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f6dcc343:::93:head 2023-11-02T15:49:34.558 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:dbbdad87:::94:head 2023-11-02T15:49:34.558 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:1cb48ce0:::95:head 2023-11-02T15:49:34.558 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:03cd461c:::96:head 2023-11-02T15:49:34.559 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:17a4ea99:::97:head 2023-11-02T15:49:34.559 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9993c9a7:::98:head 2023-11-02T15:49:34.559 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6394211c:::99:head 2023-11-02T15:49:34.559 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:94c7ae57:::100:head 2023-11-02T15:49:34.559 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6fdee5bb:::101:head 2023-11-02T15:49:34.560 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9a477fd1:::102:head 2023-11-02T15:49:34.560 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:eb850916:::103:head 2023-11-02T15:49:34.560 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:affc56b9:::104:head 2023-11-02T15:49:34.560 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b42dc814:::105:head 2023-11-02T15:49:34.560 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f319f8f0:::106:head 2023-11-02T15:49:34.561 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9a40b9de:::107:head 2023-11-02T15:49:34.561 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:8b524f28:::108:head 2023-11-02T15:49:34.561 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e3de589f:::109:head 2023-11-02T15:49:34.561 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:90f90a5b:::110:head 2023-11-02T15:49:34.561 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a7b4f1d7:::111:head 2023-11-02T15:49:34.562 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:af51766e:::112:head 2023-11-02T15:49:34.562 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b6f90bd1:::113:head 2023-11-02T15:49:34.562 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e0261208:::114:head 2023-11-02T15:49:34.562 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c9569ef7:::115:head 2023-11-02T15:49:34.562 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:61bebe50:::116:head 2023-11-02T15:49:34.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:fe93412b:::117:head 2023-11-02T15:49:34.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d3d38bee:::118:head 2023-11-02T15:49:34.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3100ba0c:::119:head 2023-11-02T15:49:34.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d0560ada:::120:head 2023-11-02T15:49:34.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f0ea8b35:::121:head 2023-11-02T15:49:34.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:766f231a:::122:head 2023-11-02T15:49:34.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a07a2582:::123:head 2023-11-02T15:49:34.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:bd7c6b3a:::124:head 2023-11-02T15:49:34.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:fb2ddaff:::125:head 2023-11-02T15:49:34.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4408e1fe:::126:head 2023-11-02T15:49:34.565 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ee1df7a7:::127:head 2023-11-02T15:49:34.565 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c3002909:::128:head 2023-11-02T15:49:34.565 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4f48ffa9:::129:head 2023-11-02T15:49:34.565 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:edf38733:::130:head 2023-11-02T15:49:34.565 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c08425c0:::131:head 2023-11-02T15:49:34.566 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:5f902d98:::132:head 2023-11-02T15:49:34.566 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:41ea2c93:::133:head 2023-11-02T15:49:34.566 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:813cee13:::134:head 2023-11-02T15:49:34.566 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:0131818d:::135:head 2023-11-02T15:49:34.566 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:26ba5a85:::136:head 2023-11-02T15:49:34.567 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:381b8a5a:::137:head 2023-11-02T15:49:34.567 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:28797e47:::138:head 2023-11-02T15:49:34.567 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:bfca7f22:::139:head 2023-11-02T15:49:34.567 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:36807075:::140:head 2023-11-02T15:49:34.567 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:80b03975:::141:head 2023-11-02T15:49:34.568 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:5c15709b:::142:head 2023-11-02T15:49:34.568 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f39ea15e:::143:head 2023-11-02T15:49:34.568 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ea992956:::144:head 2023-11-02T15:49:34.568 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:48887b1c:::145:head 2023-11-02T15:49:34.568 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:9f24a9dd:::146:head 2023-11-02T15:49:34.569 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:987f100b:::147:head 2023-11-02T15:49:34.569 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d2dd3581:::148:head 2023-11-02T15:49:34.569 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7fed1808:::149:head 2023-11-02T15:49:34.569 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c80b70e9:::150:head 2023-11-02T15:49:34.569 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:85ed90f9:::151:head 2023-11-02T15:49:34.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:36428b24:::152:head 2023-11-02T15:49:34.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d044c34a:::153:head 2023-11-02T15:49:34.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7c18bf58:::154:head 2023-11-02T15:49:34.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d1c21232:::155:head 2023-11-02T15:49:34.570 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a7a3c575:::156:head 2023-11-02T15:49:34.571 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:87da0633:::157:head 2023-11-02T15:49:34.571 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d5ac3822:::158:head 2023-11-02T15:49:34.571 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3f20522d:::159:head 2023-11-02T15:49:34.571 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6ca26563:::160:head 2023-11-02T15:49:34.571 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:532ce135:::161:head 2023-11-02T15:49:34.572 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:c78863e6:::162:head 2023-11-02T15:49:38.599 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:50.753 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.656 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:50.752 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.656 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:50.753 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.664 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:50.817 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.672 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:16:50.769 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.681 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:50.835 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.723 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:50.877 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.724 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:50.878 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:38.735 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:50.889 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.615 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:05.768 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.619 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:05.772 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.645 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:05.799 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.666 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:05.763 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.668 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:05.821 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.669 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:05.765 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:05.825 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.692 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:05.789 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:53.695 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:05.849 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.211 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:06.365 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.212 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:06.366 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.217 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:06.314 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.218 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:06.315 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.220 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:06.374 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.226 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:06.322 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.260 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:06.414 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.261 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:06.415 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:49:54.269 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:06.423 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.764 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:20.918 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.768 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:20.921 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.789 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:20.942 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.812 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:20.909 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.813 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:20.910 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.820 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:20.974 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.824 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:20.977 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.838 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:20.935 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:08.853 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:21.007 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:30.985 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.834 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:30.988 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.858 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:31.011 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.878 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:30.975 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.883 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:30.980 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.889 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:31.043 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.891 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:31.045 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:31.005 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:18.915 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:31.068 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.893 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:36.046 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.901 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:36.055 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.924 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:36.077 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.963 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:36.060 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.966 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:36.120 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.968 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:36.065 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.973 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:36.127 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.990 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:36.086 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:23.995 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:36.149 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.855 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.008 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.856 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.010 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.878 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.031 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.907 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.003 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.005 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.909 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.063 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.911 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.065 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.926 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.023 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:38.942 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.096 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.479 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.576 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.480 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.577 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.488 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:17:51.585 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.519 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.673 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.520 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.674 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.532 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:51.685 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.553 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.706 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.554 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.708 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:39.562 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:51.715 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:58.957 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.110 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:58.961 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.114 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:58.987 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.141 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.008 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.105 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.011 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.107 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.016 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.170 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.018 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.172 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.028 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.125 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.038 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.192 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.077 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:50:59.074+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:50:59.601 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.697 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.601 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.698 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.609 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:11.706 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.643 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.797 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.644 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.798 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.647 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.801 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.648 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.802 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.655 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:11.809 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:50:59.657 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:11.810 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:00.050 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:00.048+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:01.024 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:01.022+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:02.030 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:02.029+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:02.995 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:02.993+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:03.961 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:03.959+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:04.937 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:04.935+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:05.892 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:05.890+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:06.880 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:06.878+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:07.896 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:07.894+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:08.946 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:08.943+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:09.972 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:09.970+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:10.972 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:10.970+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:11.979 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:11.978+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:12.951 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:12.949+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:13.928 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:51:13.926+0000 12a62640 -1 osd.2 578 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5014 204.0 204.0 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e578) 2023-11-02T15:51:13.971 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:26.125 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:13.976 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:26.130 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:13.999 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:26.153 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.027 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:26.123 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.028 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:26.125 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.032 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:26.185 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.038 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:26.192 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.047 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:26.144 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:14.057 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:26.211 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:31.245 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.096 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:31.250 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.109 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:31.263 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.144 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:31.240 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.149 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:31.246 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.151 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:31.305 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.153 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:31.306 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.160 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:31.257 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:19.167 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:31.321 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.305 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:37.459 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.308 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:37.462 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.329 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:37.483 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.359 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:37.455 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.360 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:37.457 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.366 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:37.520 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.370 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:37.524 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.374 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:37.471 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:25.389 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:37.543 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.013 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:41.166 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.016 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:41.170 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.029 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:41.183 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.064 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:41.160 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.065 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:41.219 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.081 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:41.178 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.095 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:41.249 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.097 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:41.194 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:29.117 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:41.270 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:30.696 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7462ddf6:::163:head 2023-11-02T15:51:30.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:5d165639:::164:head 2023-11-02T15:51:30.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f43765fc:::165:head 2023-11-02T15:51:30.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b4c720e9:::166:head 2023-11-02T15:51:30.697 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e694b040:::167:head 2023-11-02T15:51:30.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:afa38db2:::168:head 2023-11-02T15:51:30.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:77ba9f53:::169:head 2023-11-02T15:51:30.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:87495034:::170:head 2023-11-02T15:51:30.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7c96bf0e:::171:head 2023-11-02T15:51:30.698 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:dbe346cc:::172:head 2023-11-02T15:51:30.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:e943ec24:::173:head 2023-11-02T15:51:30.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f97a9c0c:::174:head 2023-11-02T15:51:30.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:6f26e74d:::175:head 2023-11-02T15:51:30.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4f95e106:::176:head 2023-11-02T15:51:30.699 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:0e6f2f8f:::177:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:05db05f1:::178:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:38a78d66:::179:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:d095610b:::180:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:a1a9d709:::181:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:1e5d39db:::182:head 2023-11-02T15:51:30.700 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:f7df4fb9:::183:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:03a7f161:::184:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ba70721e:::185:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:28e5662d:::186:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:973d52de:::187:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:4303eb1c:::188:head 2023-11-02T15:51:30.701 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:b990b48e:::189:head 2023-11-02T15:51:30.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:29b8165b:::190:head 2023-11-02T15:51:30.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:3547f197:::191:head 2023-11-02T15:51:30.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:7e260936:::192:head 2023-11-02T15:51:30.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:1abec7b1:::193:head 2023-11-02T15:51:30.702 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:10fdda93:::194:head 2023-11-02T15:51:30.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:15817eea:::195:head 2023-11-02T15:51:30.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:770bab57:::196:head 2023-11-02T15:51:30.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:ed9e13e7:::197:head 2023-11-02T15:51:30.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:71471a8f:::198:head 2023-11-02T15:51:30.703 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 201:10fb1d02:::199:head 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (21157 ms) 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,0 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first is 1698940191 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,0 2023-11-02T15:51:30.704 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,0 2023-11-02T15:51:30.705 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,0 2023-11-02T15:51:30.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,1698940199,1698940200,0 2023-11-02T15:51:30.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,1698940199,1698940200,0 2023-11-02T15:51:30.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940191,1698940193,1698940194,1698940196,1698940197,1698940199,1698940200,0 2023-11-02T15:51:30.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698940194,1698940196,1698940197,1698940199,1698940200,1698940202,1698940203,0 2023-11-02T15:51:30.706 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first now 1698940194, trimmed 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (22037 ms) 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: foo0 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (17496 ms) 2023-11-02T15:51:30.707 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2023-11-02T15:51:39.075 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:51.229 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.079 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:51.232 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.114 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:51.267 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.129 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:51.226 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.131 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:51.228 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.145 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:51.299 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.151 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:51.247 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.163 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:51.317 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:39.186 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:51.339 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:56.244 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.093 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:56.247 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.105 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:18:56.259 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.136 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:56.232 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.140 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:56.237 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.140 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:56.294 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.143 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:56.297 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.152 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:18:56.249 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:44.158 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:18:56.312 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.139 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:11.292 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.143 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:11.297 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.165 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:11.318 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.191 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:11.287 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.193 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:11.290 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.201 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:11.355 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.205 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:11.359 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.209 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:11.305 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:51:59.238 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:11.392 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:03.338 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:03.336+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:04.156 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:16.309 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.158 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:16.312 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.168 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:16.322 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.206 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:16.302 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.207 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:16.304 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.217 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:16.314 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.231 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:16.385 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.232 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:16.386 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.245 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:16.398 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:04.297 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:04.294+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:05.344 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:05.342+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:06.321 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:06.318+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:07.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:07.298+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:08.280 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:08.277+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:09.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:09.309+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:10.266 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:10.264+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:11.303 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:11.301+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:12.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:12.308+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:13.322 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:13.320+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:14.370 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:14.368+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:15.346 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:15.344+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:16.371 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:16.369+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:17.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:17.345+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:18.387 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:18.384+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:19.197 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:31.351 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.201 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:31.353 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.216 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:31.370 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.251 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.347 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.253 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.350 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.260 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:31.414 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.264 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:31.418 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.271 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.368 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.282 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:31.436 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.430 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:19.427+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:19.889 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.986 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.890 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.987 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.898 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:31.995 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.989 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:32.143 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:19.990 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:32.144 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:20.001 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:32.155 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:20.013 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:32.167 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:20.014 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:32.168 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:20.023 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:32.177 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:20.452 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:20.450+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:21.418 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:21.416+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:22.389 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:22.387+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:23.367 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:23.365+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:24.326 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T15:52:24.323+0000 12a62640 -1 osd.1 586 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:5432 62.a 62:553a1aea:test-rados-api-smithi174-91559-52::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e585) 2023-11-02T15:52:34.270 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:46.424 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.272 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:46.426 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.304 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:46.458 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.321 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:46.417 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.323 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:46.420 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.327 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:46.481 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.336 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:46.490 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.348 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:46.445 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:34.361 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:46.515 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:51.641 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.502 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:51.656 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.521 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:19:51.675 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.535 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:51.631 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.536 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:51.633 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.548 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:19:51.645 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.551 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:51.705 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:51.709 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:39.568 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:19:51.722 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.345 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:01.498 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.346 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:01.500 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.366 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:01.520 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.390 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:01.487 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.395 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:01.492 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.402 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:01.556 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.404 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:01.558 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.414 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:01.511 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:49.422 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:01.576 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.367 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:11.521 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.371 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:11.525 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.394 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:11.547 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.418 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:11.515 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.421 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:11.518 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.428 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:11.582 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.430 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:11.584 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.442 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:11.539 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:52:59.451 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:11.605 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.593 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:16.747 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.600 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:16.754 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.616 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:16.769 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.655 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:16.752 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.656 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:16.753 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.667 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:16.820 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.667 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:16.764 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.668 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:16.822 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:04.679 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:16.833 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.616 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:31.769 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.625 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:31.779 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.659 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:31.813 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.675 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:31.828 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.679 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:31.775 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.680 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:31.834 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.684 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:31.781 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.710 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:31.807 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:19.729 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:31.883 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.820 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:36.974 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.823 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:36.977 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.854 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:37.007 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.884 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:36.981 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.886 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:36.983 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.888 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:37.042 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.893 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:37.047 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:37.005 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:24.922 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:37.076 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.815 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:46.968 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.817 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:46.971 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.834 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:46.988 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.864 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:46.960 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.865 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:46.962 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.867 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:47.021 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.868 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:47.022 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.882 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:46.979 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:34.895 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:47.049 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.707 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:56.861 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.710 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:56.864 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.728 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:20:56.881 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.754 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:56.851 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.757 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:56.853 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.759 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:56.913 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.764 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:56.917 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.770 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:20:56.867 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:44.780 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:20:56.934 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.876 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:02.029 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.893 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:02.047 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.911 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:02.064 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.925 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:02.022 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.928 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:02.025 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.942 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:02.039 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.953 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:02.107 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.955 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:02.108 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:49.970 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:02.124 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.739 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:06.893 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.740 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:06.894 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.751 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:06.905 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.787 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:06.883 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.788 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:06.885 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.791 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:06.945 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.793 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:06.947 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.798 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:06.895 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:53:54.804 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:06.958 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:04.926 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:17.080 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.010 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:17.164 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.037 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:17.191 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.074 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:17.228 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.075 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:17.172 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.083 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:17.237 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.084 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:17.181 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.105 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:17.259 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:05.112 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:17.209 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.800 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:21.953 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.803 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:21.957 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.821 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:21.975 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.854 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:21.951 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.857 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:21.954 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.875 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:22.029 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.876 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:21.972 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.878 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:22.032 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:09.896 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:22.050 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.942 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:37.095 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.946 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:37.100 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.966 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:37.120 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.994 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:37.090 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.995 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:37.092 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.995 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:37.149 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:24.997 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:37.151 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:25.017 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:37.114 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:25.018 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:37.172 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.869 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:42.023 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.876 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:42.029 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.890 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:42.044 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.919 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:42.016 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.922 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:42.018 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.925 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:42.078 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.927 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:42.081 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.933 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:42.030 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:29.939 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:42.093 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:44.980 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.134 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:44.983 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.137 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.008 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.162 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.035 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.131 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.189 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.036 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.133 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.037 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.190 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.061 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.158 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.062 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.216 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.644 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.741 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.645 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.742 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.647 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.801 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.648 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.802 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.655 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:21:57.751 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.657 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:57.810 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.825 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.673 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.826 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:54:45.680 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:57.834 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:04.987 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.140 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.044 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.198 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.068 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.222 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.092 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.188 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.093 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.190 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.094 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.248 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.099 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.253 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.115 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.212 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.121 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.275 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.700 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.797 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.701 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.798 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.708 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:17.805 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.722 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.876 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.723 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.877 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.726 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.879 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.727 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.881 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.732 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:17.886 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:05.736 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:17.890 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.048 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:32.202 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.098 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:32.252 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.121 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:32.275 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.151 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:32.248 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.154 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:32.251 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.170 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:32.324 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.173 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:32.270 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.174 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:32.328 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:20.198 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:32.352 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:23.773 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:55:23.766+0000 2628f640 -1 snap_mapper.add_oid found existing snaps mapped on 62:efb96199:test-rados-api-smithi174-91559-76::foo:28, removing 2023-11-02T15:55:23.782 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:55:23.775+0000 2fea7640 -1 snap_mapper.add_oid found existing snaps mapped on 62:efb96199:test-rados-api-smithi174-91559-76::foo:28, removing 2023-11-02T15:55:25.250 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:37.403 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.255 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:37.409 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.274 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:37.428 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.310 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:37.407 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.311 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:37.465 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.312 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:37.409 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.313 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:37.467 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.327 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:37.481 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:25.336 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:37.432 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.156 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:52.310 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.161 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:52.314 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.175 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:52.329 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.206 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:52.359 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.207 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:52.304 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.213 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:52.308 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.213 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:52.365 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.226 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:52.323 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:40.227 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:52.380 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.158 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:57.312 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.163 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:57.317 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.179 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:22:57.333 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.215 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:57.369 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.216 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:57.312 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.217 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:57.314 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.217 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:57.371 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.227 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:22:57.324 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:45.228 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:22:57.382 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:55:54.313 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:54.307+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:55:55.286 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:55.283+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:55:56.241 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:56.239+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:55:57.289 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:57.286+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:55:58.282 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:58.280+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:55:59.257 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:55:59.254+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:00.218 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:12.371 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.222 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:12.376 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.230 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:00.228+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:00.246 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:12.400 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.268 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:12.365 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.273 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:12.370 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.276 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:12.430 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.281 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:12.435 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.290 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:12.386 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:00.297 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:12.451 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:01.202 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:01.200+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:02.220 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:02.217+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:03.227 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:03.224+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:04.218 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:04.216+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:05.232 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:17.385 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.235 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:17.389 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.248 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:17.402 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.261 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:05.259+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:05.291 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:17.387 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.293 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:17.390 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.297 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:17.451 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.301 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:17.455 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.304 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:17.401 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:05.313 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:17.467 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:06.287 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:06.285+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:07.278 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:07.275+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:08.286 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:08.284+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:09.281 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:09.278+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:10.237 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:10.235+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:11.195 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:11.193+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:12.224 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:12.222+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:13.176 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:13.174+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:14.163 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:14.160+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:15.178 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:15.176+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:15.260 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:27.414 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.264 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:27.418 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.285 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:27.439 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.312 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:27.409 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.315 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:27.411 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.319 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:27.472 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.321 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:27.475 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.329 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:27.425 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:15.335 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:27.489 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:16.152 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:16.150+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:17.157 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:17.154+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:18.160 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:18.158+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:19.139 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:19.137+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:20.188 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:20.187+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:21.232 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:21.231+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:22.255 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:22.253+0000 12a62640 -1 osd.5 672 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:6954 217.10 217.10 (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e671) 2023-11-02T15:56:24.361 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:36.515 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.366 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:36.520 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.377 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:36.474 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.379 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:36.476 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.380 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:36.534 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.401 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:36.497 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.442 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:36.595 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.450 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:36.604 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:24.468 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:36.622 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.246 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:37.400 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.249 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:37.346 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.251 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:37.404 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.252 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:37.406 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.258 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:37.412 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.292 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:37.389 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.310 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:37.407 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.311 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:37.464 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:25.313 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:37.467 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:29.544 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:56:29.532+0000 25e93640 -1 snap_mapper.add_oid found existing snaps mapped on 62:a2d135de:test-rados-api-smithi174-91559-78::foo:29, removing 2023-11-02T15:56:29.558 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:56:29.551+0000 2b299640 -1 snap_mapper.add_oid found existing snaps mapped on 62:a2d135de:test-rados-api-smithi174-91559-78::foo:29, removing 2023-11-02T15:56:30.304 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:42.457 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.308 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:42.462 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.324 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:42.478 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.353 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:42.450 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.356 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:42.453 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.356 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:42.510 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.359 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:42.513 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.370 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:42.467 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:30.374 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:42.528 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.431 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:57.584 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.433 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:57.586 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.457 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:57.610 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.480 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:57.634 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.484 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:57.638 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.486 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:57.583 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.488 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:57.585 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.511 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:57.665 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:45.511 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:57.608 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.075 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:58.172 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.076 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:58.173 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.077 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:58.230 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:58.231 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.084 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:23:58.181 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.085 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:23:58.239 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.117 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:58.270 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.118 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:58.272 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:46.125 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:23:58.279 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:56:56.400 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:56:56.385+0000 2628f640 -1 snap_mapper.add_oid found existing snaps mapped on 219:40306fa0:test-rados-api-smithi174-91559-80::foo:2, removing 2023-11-02T15:56:56.410 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:56:56.402+0000 29a96640 -1 snap_mapper.add_oid found existing snaps mapped on 219:40306fa0:test-rados-api-smithi174-91559-80::foo:2, removing 2023-11-02T15:57:05.414 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:17.567 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.461 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:17.615 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.488 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:17.642 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.506 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:17.603 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.509 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:17.606 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.510 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:17.664 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.512 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:17.666 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.533 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:17.630 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:05.537 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:17.691 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.567 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:22.721 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.627 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:22.781 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.634 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:22.730 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.641 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:22.738 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.656 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:22.809 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.660 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:22.757 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.698 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:22.852 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.704 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:22.858 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:10.727 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:22.880 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.632 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:32.785 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.634 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:32.788 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.655 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:32.809 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.683 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:32.780 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.684 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:32.781 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.705 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:32.859 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.706 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:32.803 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.706 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:32.860 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:20.735 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:32.889 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.573 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:42.726 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.574 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:42.728 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.598 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:42.752 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.624 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:42.721 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.624 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:42.778 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.626 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:42.723 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.627 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:42.781 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.650 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:42.747 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:30.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:42.813 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:34.178 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (67867 ms) 2023-11-02T15:57:34.178 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2023-11-02T15:57:34.179 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (77057 ms) 2023-11-02T15:57:34.179 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2023-11-02T15:57:34.179 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (4548 ms) 2023-11-02T15:57:34.179 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2023-11-02T15:57:34.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (15242 ms) 2023-11-02T15:57:34.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2023-11-02T15:57:34.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (4490 ms) 2023-11-02T15:57:34.180 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2023-11-02T15:57:34.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (4918 ms) 2023-11-02T15:57:34.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2023-11-02T15:57:34.181 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (5649 ms) 2023-11-02T15:57:34.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2023-11-02T15:57:34.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (40848 ms) 2023-11-02T15:57:34.182 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2023-11-02T15:57:34.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17556 ms) 2023-11-02T15:57:34.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2023-11-02T15:57:34.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.183 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (9468 ms) 2023-11-02T15:57:34.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2023-11-02T15:57:34.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (4336 ms) 2023-11-02T15:57:34.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2023-11-02T15:57:34.184 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (26624 ms) 2023-11-02T15:57:34.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2023-11-02T15:57:34.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (17002 ms) 2023-11-02T15:57:34.185 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2023-11-02T15:57:34.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (67345 ms) 2023-11-02T15:57:34.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2023-11-02T15:57:34.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.186 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (5465 ms) 2023-11-02T15:57:34.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2023-11-02T15:57:34.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrubs... 2023-11-02T15:57:34.187 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (26157 ms) 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (12034 ms) 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2023-11-02T15:57:34.188 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (11131 ms) 2023-11-02T15:57:34.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2023-11-02T15:57:34.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T15:57:34.189 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (11285 ms) 2023-11-02T15:57:34.190 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2023-11-02T15:57:35.577 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:47.731 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.580 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:47.734 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.603 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:24:47.757 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.627 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:47.723 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.628 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:47.725 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.640 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:47.794 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.641 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:24:47.738 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.645 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:47.799 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:35.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:24:47.813 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.718 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:02.872 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.725 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:02.879 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:02.906 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.771 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:02.868 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.772 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:02.869 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.784 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:02.938 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.786 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:02.940 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.800 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:02.897 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:57:50.817 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:02.971 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.682 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:22.836 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.686 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:22.840 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.711 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:22.865 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.736 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:22.833 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.740 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:22.837 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.753 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:22.907 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.756 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:22.910 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.757 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:22.854 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:10.781 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:22.934 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.360 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:23.457 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.361 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:23.458 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.370 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:23.467 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.401 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:23.555 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.402 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:23.556 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.405 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:23.559 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:23.562 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.413 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:23.567 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:11.419 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:23.573 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:17.146 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:17.144+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:18.185 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:18.183+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:19.172 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:19.170+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:20.206 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:20.205+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:21.234 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:21.231+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:22.247 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:22.245+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:23.283 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:23.281+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:24.256 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:24.254+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:25.278 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:25.276+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:25.717 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:37.870 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.721 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:37.875 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.743 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:37.897 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.769 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:37.866 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.771 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:37.925 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.771 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:37.868 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.773 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:37.927 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.789 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:37.885 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:25.790 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:37.944 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:26.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:26.288+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:27.293 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:27.291+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:28.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:28.310+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:29.325 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:29.323+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:30.351 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:30.349+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:30.778 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:42.931 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.825 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:42.979 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.836 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:42.990 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.872 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:42.968 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.872 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:42.969 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.874 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:43.028 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.876 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:43.030 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.883 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:42.980 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:30.887 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:43.040 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:31.321 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:31.320+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:32.345 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:32.342+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:33.354 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:33.352+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:34.332 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:34.330+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:35.326 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:35.324+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:36.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:36.279+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:37.275 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:37.273+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:38.255 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:38.252+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:39.212 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:39.210+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:40.204 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:40.202+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:40.766 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:52.920 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.769 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:52.922 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.790 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:52.944 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.817 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:52.914 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.819 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:52.916 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.824 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:52.978 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.828 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:52.982 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.833 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:52.930 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:40.843 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:52.997 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:41.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:41.225+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:42.180 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:42.179+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:43.173 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:43.170+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:44.151 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:44.149+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:45.146 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:45.144+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:46.172 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:46.169+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:46.678 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:58.832 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.683 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:58.836 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.704 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:58.858 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.730 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:58.827 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.732 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:58.829 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.740 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:58.894 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.743 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:58.897 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.746 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:25:58.843 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:46.758 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:58.912 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:47.176 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:47.174+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:48.188 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:48.186+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:49.225 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:49.222+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:50.259 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:50.256+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:50.792 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:02.946 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.797 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:02.950 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.808 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:02.962 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.844 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:02.998 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.845 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:02.942 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.847 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:03.001 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.848 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:02.945 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.857 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:02.954 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:50.861 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:03.015 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:51.290 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:51.289+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:52.259 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:52.257+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:53.271 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:53.269+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:54.311 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:54.309+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:55.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:55.322+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:55.813 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:07.967 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.816 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:07.970 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.828 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:07.982 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.864 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:07.961 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.867 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:07.964 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.876 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:08.030 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.877 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:07.974 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.880 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:08.034 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:55.896 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:08.050 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:58:56.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:56.322+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:57.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:57.322+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:58.298 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:58.295+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:58:59.306 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:58:59.303+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:00.326 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:00.322+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:01.294 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:01.291+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:02.256 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:02.253+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:03.292 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:03.289+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:04.257 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:04.254+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:05.224 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:05.220+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:05.968 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:59:05.966+0000 12a62640 -1 osd.7 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.593911+0000 front 2023-11-02T15:57:41.592092+0000 (oldest deadline 2023-11-02T15:59:05.082714+0000) 2023-11-02T15:59:06.216 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:06.214+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:06.661 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:59:06.654+0000 12a62640 -1 osd.4 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.688687+0000 front 2023-11-02T15:57:41.692008+0000 (oldest deadline 2023-11-02T15:59:05.780743+0000) 2023-11-02T15:59:06.944 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:59:06.942+0000 12a62640 -1 osd.7 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.593911+0000 front 2023-11-02T15:57:41.592092+0000 (oldest deadline 2023-11-02T15:59:05.082714+0000) 2023-11-02T15:59:07.265 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:07.262+0000 12a62640 -1 osd.2 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:43.246850+0000 front 2023-11-02T15:57:43.183715+0000 (oldest deadline 2023-11-02T15:59:06.643219+0000) 2023-11-02T15:59:07.266 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:07.263+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:07.382 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T15:59:07.375+0000 12a62640 -1 osd.0 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:42.571533+0000 front 2023-11-02T15:57:42.580032+0000 (oldest deadline 2023-11-02T15:59:06.670192+0000) 2023-11-02T15:59:07.700 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T15:59:07.698+0000 12a62640 -1 osd.4 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.688687+0000 front 2023-11-02T15:57:41.692008+0000 (oldest deadline 2023-11-02T15:59:05.780743+0000) 2023-11-02T15:59:07.745 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T15:59:07.743+0000 12a62640 -1 osd.6 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:42.841189+0000 front 2023-11-02T15:57:42.844579+0000 (oldest deadline 2023-11-02T15:59:06.931681+0000) 2023-11-02T15:59:07.957 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T15:59:07.955+0000 12a62640 -1 osd.7 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.593911+0000 front 2023-11-02T15:57:41.592092+0000 (oldest deadline 2023-11-02T15:59:05.082714+0000) 2023-11-02T15:59:08.074 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T15:59:08.072+0000 12a62640 -1 osd.5 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:41.643612+0000 front 2023-11-02T15:57:41.644923+0000 (oldest deadline 2023-11-02T15:59:07.569235+0000) 2023-11-02T15:59:08.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:08.293+0000 12a62640 -1 osd.2 728 heartbeat_check: no reply from 172.21.15.174:6824 osd.3 since back 2023-11-02T15:57:43.246850+0000 front 2023-11-02T15:57:43.183715+0000 (oldest deadline 2023-11-02T15:59:06.643219+0000) 2023-11-02T15:59:08.303 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:08.300+0000 12a62640 -1 osd.2 728 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:09.334 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T15:59:09.300+0000 12a62640 -1 osd.2 730 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:7755 225.1d 225.5dd3f7bd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e725) 2023-11-02T15:59:10.987 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:23.141 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:10.991 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:23.145 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.017 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:23.171 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.056 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:23.153 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.057 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:23.211 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.059 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:23.156 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.060 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:23.214 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.080 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:23.177 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:11.085 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:23.238 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:15.410 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T15:59:15.408+0000 14265640 -1 osd.3 734 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T15:59:25.983 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:38.136 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:25.985 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:38.138 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.013 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:38.167 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.031 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:38.127 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.033 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:38.130 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.047 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:38.201 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.050 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:38.204 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.060 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:38.157 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:26.081 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:38.234 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:30.997 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:43.151 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.000 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:43.153 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.014 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:43.167 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.048 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:43.145 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.049 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:43.203 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.050 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:43.147 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.054 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:43.207 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.060 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:43.157 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:31.065 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:43.218 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.285 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:58.438 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.298 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:58.451 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.329 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:26:58.481 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.340 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:58.436 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.345 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:58.442 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.364 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:58.518 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.369 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:58.523 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.373 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:26:58.470 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:46.394 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:26:58.548 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.143 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:03.296 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.147 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:03.301 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.164 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:03.318 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.199 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:03.353 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.199 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:03.296 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.200 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:03.354 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.201 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:03.298 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.214 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:03.311 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T15:59:51.216 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:03.370 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.211 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:18.365 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.216 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:18.370 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.238 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:18.392 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.260 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:18.357 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.264 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:18.361 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.273 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:18.427 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.275 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:18.429 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.284 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:18.381 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:06.295 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:18.449 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.224 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:23.378 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.229 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:23.383 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.261 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:23.415 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.280 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:23.377 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.282 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:23.379 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.283 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:23.436 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.284 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:23.438 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.292 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:23.389 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:11.297 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:23.451 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:18.173 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:18.171+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:19.147 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:19.145+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:20.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:20.140+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:21.100 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:21.097+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:22.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:22.140+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:23.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:23.108+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:24.122 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:24.120+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:25.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:25.127+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:26.096 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:26.093+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:26.311 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:38.464 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.315 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:38.469 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.331 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:38.485 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.358 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:38.455 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.361 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:38.515 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.362 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:38.459 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.364 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:38.518 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.383 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:38.537 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:26.383 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:38.480 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:27.079 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:27.076+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:28.070 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:28.067+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:29.087 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:29.084+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:30.072 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:30.070+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:31.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:31.023+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:31.322 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:43.476 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.326 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:43.480 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.341 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:43.494 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.372 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:43.469 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.374 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:43.528 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.375 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:43.472 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.378 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:43.532 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.390 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:43.544 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.391 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:43.488 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:31.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:31.995+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:33.008 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:33.005+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:34.018 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:34.016+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:35.046 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:35.044+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:36.003 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:36.002+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:36.970 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:36.969+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:37.930 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:37.928+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:38.935 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:38.933+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:39.915 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:39.913+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:40.899 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:40.897+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:41.353 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:53.507 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.355 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:53.509 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.373 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:27:53.527 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.405 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:53.501 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:53.561 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.408 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:53.505 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.409 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:53.563 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.424 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:27:53.521 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.425 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:27:53.579 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:41.879 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:41.877+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:42.842 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:42.840+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:43.862 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:43.859+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:44.832 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:44.830+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:45.881 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:45.879+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:46.842 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:46.840+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:47.891 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:47.890+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:48.856 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:48.854+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:49.829 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:49.827+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:50.834 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:50.831+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:51.382 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:03.535 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.386 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:03.539 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.402 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:03.556 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.429 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:03.526 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.436 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:03.533 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.436 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:03.590 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.440 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:03.594 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.450 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:03.547 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.455 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:03.609 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:51.816 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:51.814+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:52.845 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:52.843+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:53.834 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:53.832+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:54.839 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:54.837+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:55.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:55.793+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:56.396 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:08.550 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.398 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:08.552 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.411 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:08.565 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.446 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:08.543 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.452 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:08.549 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.461 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:08.615 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.463 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:08.560 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.464 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:08.618 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.476 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:08.630 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:00:56.845 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:56.843+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:57.872 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:57.871+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:58.916 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:58.914+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:00:59.951 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:00:59.949+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:01:00.922 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:01:00.920+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:01:01.898 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:01:01.896+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:01:02.935 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:01:02.933+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:01:03.975 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:01:03.973+0000 12a62640 -1 osd.1 748 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4556.0:8155 62.8 62:13024573:test-rados-api-smithi174-91559-97::foo:head [writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e747) 2023-11-02T16:01:11.457 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:23.611 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.463 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:23.617 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:23.641 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.517 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:23.613 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.517 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:23.671 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.520 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:23.617 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.521 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:23.675 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.539 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:23.636 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:11.540 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:23.694 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.161 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:24.258 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.162 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:24.259 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.169 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:24.322 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.170 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:24.324 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.171 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:24.268 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.179 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:24.333 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.204 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:24.358 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.205 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:24.359 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:12.218 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:24.372 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:16.234 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:01:16.226+0000 2ca9c640 -1 snap_mapper.add_oid found existing snaps mapped on 62:13024573:test-rados-api-smithi174-91559-97::foo:31, removing 2023-11-02T16:01:16.235 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:01:16.208+0000 26a92640 -1 snap_mapper.add_oid found existing snaps mapped on 62:13024573:test-rados-api-smithi174-91559-97::foo:31, removing 2023-11-02T16:01:17.669 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:29.823 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.738 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:29.892 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.754 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:29.908 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.795 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:29.892 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.799 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:29.896 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.810 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:29.964 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.812 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:29.966 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.814 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:29.911 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:17.827 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:29.981 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.609 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:38.763 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.632 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:38.786 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.668 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:38.822 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.669 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:38.766 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.672 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:38.769 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.688 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:38.842 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.689 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:38.843 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.700 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:38.797 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:26.712 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:38.866 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.510 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:43.664 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.515 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:43.668 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.530 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:43.684 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.563 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:43.716 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.563 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:43.660 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.566 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:43.719 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.566 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:43.663 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.582 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:43.678 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:31.582 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:43.736 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.544 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:53.698 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.549 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:53.703 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.575 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:28:53.728 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.598 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:53.695 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.599 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:53.753 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.601 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:53.698 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.603 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:53.755 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.625 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:28:53.721 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:41.627 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:28:53.781 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.697 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:03.850 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.701 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:03.855 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.735 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:03.888 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.755 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:03.852 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.756 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:03.853 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.762 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:03.916 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.765 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:03.918 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.781 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:03.878 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:51.787 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:03.941 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.610 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:08.763 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.611 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:08.765 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.625 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:08.779 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.655 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:08.752 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.657 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:08.754 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:08.813 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.660 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:08.814 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.675 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:08.828 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:01:56.676 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:08.772 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.848 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:24.002 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.857 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:24.011 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.863 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:23.960 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.866 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:23.963 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.866 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:24.020 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.870 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:24.024 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.974 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:24.127 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.983 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:24.137 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:11.991 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:24.088 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.873 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:34.026 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.875 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:34.029 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.905 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:34.058 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.920 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:34.016 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.921 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:34.018 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.944 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:34.041 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.951 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:34.105 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.956 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:34.110 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:21.989 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:34.143 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.807 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:38.960 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.812 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:38.966 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.830 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:38.984 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.861 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:38.958 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.864 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:38.961 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.866 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:39.019 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.869 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:39.023 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.879 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:38.975 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:26.884 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:39.038 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:36.974 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:49.128 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:36.979 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:49.133 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.001 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:29:49.155 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.034 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:49.188 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:49.189 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.038 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:49.135 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.057 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:49.154 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.069 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:29:49.222 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:37.080 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:29:49.177 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.941 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:04.094 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.943 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:04.096 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.963 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:04.116 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.987 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:04.084 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.992 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:04.088 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.995 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:04.149 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:51.996 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:04.150 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:52.011 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:04.108 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:52.016 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:04.170 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:56.954 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:09.107 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:56.956 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:09.110 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:56.971 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:09.125 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.003 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:09.100 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.006 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:09.160 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.006 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:09.103 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.009 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:09.163 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.016 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:09.113 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:02:57.020 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:09.174 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:11.056 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:11.054+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:11.991 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:24.144 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:11.995 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:24.149 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.010 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:24.164 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.041 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:24.138 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.044 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:24.140 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.044 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:24.198 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.048 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:24.201 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.060 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:24.157 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:12.067 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:12.065+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:12.070 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:24.224 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:13.019 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:13.017+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:14.038 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:14.036+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:15.056 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:15.053+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:16.099 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:16.097+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:17.004 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:29.158 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.008 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:29.162 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.021 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:29.175 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.056 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:29.152 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.057 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:29.211 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.057 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:29.154 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.061 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:29.214 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.072 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:29.169 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.074 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:29.228 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:17.147 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:17.144+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:18.156 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:18.153+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:19.167 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:19.164+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:20.066 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:32.220 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:32.231 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.113 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:32.267 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.122 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:32.218 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.138 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:32.292 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.144 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:32.241 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.149 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:32.303 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.161 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:32.258 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.164 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:32.317 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:20.190 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:20.186+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:21.157 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:03:21.153+0000 12a62640 -1 osd.3 832 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:108 237.6s0 237:69df9a62:test-rados-api-smithi174-91559-115::foo:head [writefull 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e831) 2023-11-02T16:03:27.173 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:39.326 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.176 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:39.330 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.193 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:39.347 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.225 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:39.322 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.226 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:39.323 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.226 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:39.380 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.228 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:39.381 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.245 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:39.399 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:27.246 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:39.343 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.041 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:49.194 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.088 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:49.242 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.089 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:49.186 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.091 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:49.188 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.106 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:49.202 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.107 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:49.261 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.151 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:49.305 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.153 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:49.307 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:37.166 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:49.319 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.117 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:59.271 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.122 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:59.276 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.139 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:30:59.292 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.168 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:59.264 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.169 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:59.266 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.182 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:30:59.279 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.183 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:59.337 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.187 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:59.341 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:47.200 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:30:59.354 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:54.870 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:54.867+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:03:55.887 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:55.886+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:03:56.856 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:56.854+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:03:57.095 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:09.249 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.144 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:09.298 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.160 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:09.314 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.190 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:09.344 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.191 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:09.288 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.194 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:09.291 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.196 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:09.350 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.207 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:09.304 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.210 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:09.363 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:03:57.881 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:57.879+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:03:58.856 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:58.854+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:03:59.882 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:03:59.879+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:04:00.017 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:04:00.016+0000 12a62640 -1 osd.5 836 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:02:39.499058+0000 front 2023-11-02T16:02:39.502607+0000 (oldest deadline 2023-11-02T16:03:59.984868+0000) 2023-11-02T16:04:00.798 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:04:00.796+0000 12a62640 -1 osd.6 836 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:02:37.103856+0000 front 2023-11-02T16:02:37.110788+0000 (oldest deadline 2023-11-02T16:04:00.593817+0000) 2023-11-02T16:04:00.904 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:04:00.900+0000 12a62640 -1 osd.0 836 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:02:34.153131+0000 front 2023-11-02T16:02:34.153760+0000 (oldest deadline 2023-11-02T16:04:00.036952+0000) 2023-11-02T16:04:00.904 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:04:00.901+0000 12a62640 -1 osd.0 836 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:04:01.044 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:04:01.042+0000 12a62640 -1 osd.5 836 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:02:39.499058+0000 front 2023-11-02T16:02:39.502607+0000 (oldest deadline 2023-11-02T16:03:59.984868+0000) 2023-11-02T16:04:01.894 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:04:01.892+0000 12a62640 -1 osd.0 837 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:109 239.16 239.4659fb96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e835) 2023-11-02T16:04:02.335 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:14.489 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.338 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:14.492 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.353 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:14.507 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.392 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:14.489 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:14.562 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.410 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:14.564 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.412 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:14.509 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.425 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:14.522 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:02.428 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:14.582 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:10.227 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:10.223+0000 14265640 -1 osd.7 844 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T16:04:17.193 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:29.346 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.239 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:29.393 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.269 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:29.423 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.287 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:29.384 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.290 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:29.387 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.291 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:29.445 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.294 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:29.448 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.318 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:29.415 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:17.323 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:29.477 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.036 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:30.132 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.037 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:30.133 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.044 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:30.141 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.114 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:30.268 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.115 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:30.269 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.116 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:30.270 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.117 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:30.271 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.123 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:30.277 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:18.125 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:30.279 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.288 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:49.441 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.292 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:49.446 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.312 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:49.466 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.343 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:49.439 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.345 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:49.442 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.361 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:49.515 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.363 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:49.460 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.364 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:49.518 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:37.385 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:49.538 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.017 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:50.114 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.018 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:50.115 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.026 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:31:50.123 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:50.245 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.092 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:50.246 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.094 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:50.248 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.096 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:50.249 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.101 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:31:50.255 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:38.104 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:31:50.258 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:42.980 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:42.977+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:44.000 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:43.998+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:45.050 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:45.047+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:46.061 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:46.058+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:47.032 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:47.030+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:48.016 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:48.014+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:48.978 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:48.975+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:50.015 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:50.012+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:51.047 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:51.045+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:52.042 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:52.041+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:52.334 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:04.488 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.336 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:04.490 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.354 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:04.507 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.382 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:04.479 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.383 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:04.537 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.384 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:04.481 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.388 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:04.541 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.401 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:04.498 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:52.405 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:04.559 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:53.050 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:53.048+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:54.068 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:54.066+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:55.115 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:55.113+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:56.164 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:56.163+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:57.139 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:57.138+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:57.390 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:09.543 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.392 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:09.546 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.402 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:09.556 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.441 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:09.537 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.441 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:09.538 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.444 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:09.597 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.445 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:09.599 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.452 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:09.549 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:57.455 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:09.609 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:04:58.091 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:58.089+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:04:59.092 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:04:59.091+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:00.103 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:00.101+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:01.067 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:01.065+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:02.024 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:02.023+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:02.997 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:02.996+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:04.038 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:04.037+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:05.052 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:05.050+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:06.061 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:06.060+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:07.106 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:07.104+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:07.397 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:19.550 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.398 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:19.551 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.411 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:19.564 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.445 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:19.542 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.446 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:19.600 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.448 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:19.545 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.451 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:19.605 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.463 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:19.560 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:07.469 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:19.622 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:08.140 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:08.137+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:09.152 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:09.149+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:10.167 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:10.164+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:11.151 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:11.149+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:12.195 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:12.193+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:13.147 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:13.145+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:14.121 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:14.119+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:15.128 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:15.125+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:16.124 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:16.122+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:17.103 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:17.100+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:17.480 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:29.633 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.484 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:29.638 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.506 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:29.660 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.531 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:29.628 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.534 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:29.631 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.536 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:29.689 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.539 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:29.693 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.548 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:29.645 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:17.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:29.710 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:18.084 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:18.082+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:19.121 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:19.119+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:20.084 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:20.081+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:21.088 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:21.085+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:22.045 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:22.043+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:22.491 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:34.644 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.495 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:34.649 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.512 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:34.666 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.542 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:34.639 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.545 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:34.699 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.546 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:34.642 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.550 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:34.703 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.558 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:34.654 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:22.565 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:34.719 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:23.045 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:23.042+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:24.080 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:24.077+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:25.070 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:25.067+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:26.098 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:26.096+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:27.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:05:27.049+0000 12a62640 -1 osd.7 848 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:256 240.19 240.ea6c9659 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e848) 2023-11-02T16:05:37.552 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:49.705 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.555 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:49.709 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.576 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:49.729 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.603 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:49.699 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.606 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:49.703 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.607 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:49.761 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.609 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:49.762 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.625 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:49.721 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:37.628 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:49.781 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.279 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:50.375 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.279 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:50.376 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.287 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:32:50.384 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.310 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:50.462 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.311 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:50.463 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.317 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:50.471 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.327 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:50.481 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.329 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:50.482 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:38.339 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:50.493 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.518 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:04.672 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.522 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:04.676 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.575 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:04.672 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.577 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:04.674 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.580 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:04.733 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.581 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:04.735 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.583 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:04.737 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.606 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:04.702 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:52.612 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:04.766 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.358 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:05.454 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.358 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:05.455 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.370 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:05.467 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.488 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:05.642 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.489 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:05.643 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.503 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:05.656 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.509 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:05.663 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.510 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:05.664 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:05:53.520 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:05.673 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:14.985 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.836 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:14.990 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.861 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:15.015 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.885 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:14.982 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.886 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:15.040 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.890 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:15.043 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.890 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:14.987 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.915 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:15.012 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:02.920 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:15.074 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.843 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:19.996 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.847 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:20.000 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.865 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:20.018 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.895 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:19.991 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.896 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:19.993 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.901 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:20.055 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.902 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:20.056 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:20.004 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:07.914 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:20.068 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.746 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:34.899 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.750 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:34.903 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.771 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:34.924 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.800 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:34.896 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.803 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:34.900 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.807 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:34.961 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.808 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:34.962 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.819 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:34.916 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:22.831 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:34.985 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.754 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:39.907 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.755 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:39.909 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.767 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:39.921 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.800 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:39.897 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.803 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:39.957 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.804 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:39.900 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.806 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:39.960 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.814 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:39.911 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:27.819 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:39.973 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:34.175 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:34.173+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:35.217 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:35.214+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:36.247 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:36.244+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:37.200 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:37.198+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:38.218 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:38.217+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:39.254 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:39.252+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:40.282 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:40.280+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:41.286 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:41.284+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:42.294 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:42.291+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:42.796 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:54.950 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.798 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:54.952 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.813 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:54.966 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.847 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:54.943 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.848 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:54.944 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.849 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.003 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.850 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.004 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.870 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:54.966 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:42.872 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.026 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.346 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:43.344+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:43.528 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:55.624 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.529 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:55.626 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.540 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:33:55.636 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.540 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:55.694 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.542 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:55.696 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.552 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:33:55.706 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.555 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.709 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.710 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:43.565 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:33:55.719 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:06:44.321 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:44.319+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:45.344 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:45.342+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:46.344 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:46.342+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:47.350 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:47.346+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:48.334 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:48.332+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:49.373 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:49.371+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:50.358 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:50.356+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:51.335 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:51.333+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:52.295 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:52.293+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:53.297 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:53.293+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:54.347 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:54.345+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:55.388 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:55.386+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:56.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:56.429+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:57.451 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:57.449+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:58.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:58.435+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:06:59.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:06:59.474+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:00.523 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:00.521+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:01.561 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:01.559+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:02.527 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:02.516+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:02.855 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:15.008 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.858 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:15.012 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.881 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:15.032 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.908 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:15.005 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.909 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:15.063 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.909 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:15.006 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.913 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:15.066 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.928 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:15.024 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:02.934 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:15.085 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:03.483 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:03.482+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:04.465 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:04.463+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:05.447 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:05.445+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:06.494 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:06.492+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:07.460 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:07.458+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:07.869 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:20.022 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.870 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:20.024 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.883 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:20.037 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.914 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:20.010 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.918 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:20.015 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.929 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:20.026 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.931 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:20.084 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.935 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:20.089 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:07.950 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:20.104 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:08.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:08.468+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:09.455 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:09.452+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:10.494 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:10.492+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:11.534 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:11.532+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:12.489 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:12.487+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:13.535 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:13.533+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:14.552 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:14.550+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:15.524 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:15.522+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:16.520 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:16.518+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:17.537 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:17.535+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:18.568 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:18.566+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:19.605 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:19.603+0000 12a62640 -1 osd.3 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.033950+0000 front 2023-11-02T16:05:54.042418+0000 (oldest deadline 2023-11-02T16:07:19.221059+0000) 2023-11-02T16:07:19.606 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:19.604+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:19.770 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:07:19.767+0000 12a62640 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.310355+0000 front 2023-11-02T16:05:54.308136+0000 (oldest deadline 2023-11-02T16:07:19.597409+0000) 2023-11-02T16:07:20.640 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:20.637+0000 12a62640 -1 osd.3 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.033950+0000 front 2023-11-02T16:05:54.042418+0000 (oldest deadline 2023-11-02T16:07:19.221059+0000) 2023-11-02T16:07:20.640 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:20.638+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:20.676 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:07:20.675+0000 12a62640 -1 osd.1 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.312798+0000 front 2023-11-02T16:05:54.319427+0000 (oldest deadline 2023-11-02T16:07:20.199404+0000) 2023-11-02T16:07:20.750 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:07:20.747+0000 12a62640 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.310355+0000 front 2023-11-02T16:05:54.308136+0000 (oldest deadline 2023-11-02T16:07:19.597409+0000) 2023-11-02T16:07:21.357 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:07:21.354+0000 12a62640 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:57.845479+0000 front 2023-11-02T16:05:57.846414+0000 (oldest deadline 2023-11-02T16:07:21.339703+0000) 2023-11-02T16:07:21.676 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:21.672+0000 12a62640 -1 osd.3 870 heartbeat_check: no reply from 172.21.15.175:6804 osd.7 since back 2023-11-02T16:05:54.033950+0000 front 2023-11-02T16:05:54.042418+0000 (oldest deadline 2023-11-02T16:07:19.221059+0000) 2023-11-02T16:07:21.678 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:21.676+0000 12a62640 -1 osd.3 870 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:22.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:07:22.708+0000 12a62640 -1 osd.3 872 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.15765.0:493 242.1e 242.1e (undecoded) ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e869) 2023-11-02T16:07:22.925 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:35.079 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.928 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:35.082 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.949 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:35.103 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.980 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:35.134 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.981 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.078 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.983 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:35.137 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:22.984 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.081 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.003 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.100 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.004 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:35.158 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.760 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.856 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.761 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.858 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.770 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:35.867 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.891 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:36.044 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.892 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:36.045 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.904 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:36.058 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.927 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:36.081 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.928 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:36.082 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:23.937 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:36.091 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:25.071 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:07:25.069+0000 14265640 -1 osd.7 874 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T16:07:37.973 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:50.127 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:37.978 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:50.132 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.016 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:34:50.169 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.025 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:50.122 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.028 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:50.124 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.030 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:50.184 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.033 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:50.186 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.058 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:34:50.155 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:38.063 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:34:50.216 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.149 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:05.302 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.155 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:05.308 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.190 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:05.344 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.217 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:05.313 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.218 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:05.371 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.218 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:05.314 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.220 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:05.373 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.250 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:05.404 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:53.254 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:05.351 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:07:54.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.481 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (11167 ms) 2023-11-02T16:07:54.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2023-11-02T16:07:54.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (90022 ms) 2023-11-02T16:07:54.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2023-11-02T16:07:54.482 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (6497 ms) 2023-11-02T16:07:54.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2023-11-02T16:07:54.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25931 ms) 2023-11-02T16:07:54.483 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2023-11-02T16:07:54.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (92183 ms) 2023-11-02T16:07:54.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2023-11-02T16:07:54.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12127 ms) 2023-11-02T16:07:54.484 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2023-11-02T16:07:54.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2023-11-02T16:07:54.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000010 69 73 |is| 2023-11-02T16:07:54.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000012 2023-11-02T16:07:54.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12272 ms) 2023-11-02T16:07:54.485 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2023-11-02T16:07:54.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = reef 2023-11-02T16:07:54.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6658 ms) 2023-11-02T16:07:54.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (1528222 ms total) 2023-11-02T16:07:54.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2023-11-02T16:07:54.486 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2023-11-02T16:07:54.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2023-11-02T16:07:54.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1444 ms) 2023-11-02T16:07:54.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2023-11-02T16:07:54.487 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (12148 ms) 2023-11-02T16:07:54.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2023-11-02T16:07:54.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (17714 ms) 2023-11-02T16:07:54.488 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2023-11-02T16:07:54.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (17 ms) 2023-11-02T16:07:54.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (31323 ms total) 2023-11-02T16:07:54.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2023-11-02T16:07:54.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2023-11-02T16:07:54.489 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2023-11-02T16:07:54.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7619 ms) 2023-11-02T16:07:54.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2023-11-02T16:07:54.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (88041 ms) 2023-11-02T16:07:54.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2023-11-02T16:07:54.490 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrub... 2023-11-02T16:07:54.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2023-11-02T16:07:54.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (100475 ms) 2023-11-02T16:07:54.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2023-11-02T16:07:54.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10714 ms) 2023-11-02T16:07:54.491 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2023-11-02T16:07:54.492 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (88321 ms) 2023-11-02T16:07:54.492 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2023-11-02T16:07:54.492 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8137 ms) 2023-11-02T16:07:54.492 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2023-11-02T16:07:54.492 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (11516 ms) 2023-11-02T16:07:54.493 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2023-11-02T16:08:03.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:15.231 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.081 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:15.235 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.103 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:15.256 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.128 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:15.224 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.129 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:15.225 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.130 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:15.284 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.134 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:15.287 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.152 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:15.248 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:03.155 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:15.308 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.092 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:20.245 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.094 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:20.248 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.109 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:20.262 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.139 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:20.235 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.142 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:20.238 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.145 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:20.299 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.148 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:20.301 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.157 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:20.254 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:08.160 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:20.314 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.363 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:21.516 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.366 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:21.520 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.381 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:21.534 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.417 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:21.514 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.418 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:21.515 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.431 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:21.584 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.431 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:21.528 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.433 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:21.587 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:09.447 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:21.601 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.230 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:35.383 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.233 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:35.387 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.280 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:35.376 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.281 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:35.378 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.295 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:35.449 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.297 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:35.450 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.313 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:35.467 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.337 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:35.433 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:23.362 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:35.516 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.368 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:36.464 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.369 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:36.465 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.377 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:36.474 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.517 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:36.671 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.518 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:36.672 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.527 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:36.681 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.801 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:36.955 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.803 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:36.956 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:24.824 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:36.978 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.350 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:50.503 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.352 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:50.505 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.380 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:50.534 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.401 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:50.498 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.402 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:50.556 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.403 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:50.500 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.404 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:50.557 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.434 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:50.588 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:38.436 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:50.533 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.243 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:55.396 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.248 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:55.401 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.266 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:35:55.419 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.294 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:55.390 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.295 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:55.392 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.304 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:55.457 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.306 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:55.459 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.309 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:35:55.405 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:43.321 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:35:55.475 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.262 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:05.415 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.278 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:05.432 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.306 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:05.459 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.311 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:05.408 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.314 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:05.411 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.315 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:05.469 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.335 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:05.489 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.338 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:05.435 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:08:53.372 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:05.526 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.286 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:15.439 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.288 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:15.442 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.316 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:15.470 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.337 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:15.433 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.339 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:15.436 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.348 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:15.501 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.363 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:15.459 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.373 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:15.527 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:03.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:15.561 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.300 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:20.454 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.302 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:20.456 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.322 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:20.476 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.350 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:20.503 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.355 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:20.509 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.357 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:20.453 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.358 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:20.455 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.374 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:20.526 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:08.374 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:20.471 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.322 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:35.474 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.372 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:35.526 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.396 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:35.550 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.424 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:35.520 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.424 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:35.521 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.440 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:35.594 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.441 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:35.595 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.445 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:35.542 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:23.482 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:35.635 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.142 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:36.239 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.144 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:36.241 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.152 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:36.249 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.233 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:36.387 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.234 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:36.388 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.234 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:36.388 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.236 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:36.389 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.244 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:36.397 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:24.244 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:36.398 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.414 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:50.567 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.461 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:50.558 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.462 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:50.616 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.464 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:50.561 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.495 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:36:50.648 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.495 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:36:50.589 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.515 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:50.669 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.517 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:50.671 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:38.547 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:36:50.700 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.483 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:00.634 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.483 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:00.637 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.509 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:00.662 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.533 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:00.630 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.534 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:00.687 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.535 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:00.631 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.537 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:00.691 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.556 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:00.653 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:48.561 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:00.714 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.570 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:05.721 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.572 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:05.725 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.579 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:05.676 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.582 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:05.735 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.632 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:05.728 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.632 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:05.786 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.648 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:05.744 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.677 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:05.830 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:09:53.694 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:05.847 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:20.905 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.755 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:20.908 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.777 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:20.930 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.806 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:20.959 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.807 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:20.960 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.814 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:20.911 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.818 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:20.915 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.831 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:20.985 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:08.845 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:20.941 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.485 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:23.636 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:23.640 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.506 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:23.658 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.537 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:23.633 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.538 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:23.635 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.539 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:23.693 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.544 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:23.697 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.553 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:23.649 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:11.559 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:23.712 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.699 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:35.852 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.702 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:35.855 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.723 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:35.876 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.753 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:35.849 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.754 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:35.850 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.765 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:35.918 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.769 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:35.922 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.775 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:35.871 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:23.795 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:35.945 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.860 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:41.013 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.863 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:41.016 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.870 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:41.023 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.871 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:40.968 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.916 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:41.013 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.926 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:41.080 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.930 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:41.026 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.936 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:41.090 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:28.945 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:41.099 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.800 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:50.953 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.803 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:50.957 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.826 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:50.979 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.850 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:50.946 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.854 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:51.007 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.855 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:50.951 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.858 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:51.012 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.874 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:37:50.971 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:38.879 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:51.033 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.830 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:00.984 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:00.986 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.848 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:01.001 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.877 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:00.974 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.883 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:00.979 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.895 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:00.991 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.898 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:01.052 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.901 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:01.055 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:48.914 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:01.067 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.810 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:10.963 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.857 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:11.010 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.872 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:11.025 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.903 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:11.000 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.905 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:11.058 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.906 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:11.003 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.906 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:11.060 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.918 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:11.014 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:10:58.918 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:11.072 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.836 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:20.990 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.883 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:21.036 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.898 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:21.051 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.932 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:21.028 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.933 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:21.085 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.934 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:21.029 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.934 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:21.087 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.945 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:21.098 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:08.946 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:21.042 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.908 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:31.061 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.912 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:31.065 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.927 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:31.080 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.961 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:31.057 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.967 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:31.059 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.967 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:31.115 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.968 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:31.118 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.973 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:31.070 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:18.977 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:31.130 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.936 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:41.089 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.938 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:41.091 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.952 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:41.105 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.984 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:41.081 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.986 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:41.139 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.987 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:41.084 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:28.989 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:41.142 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:29.000 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:41.096 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:29.001 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:41.154 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:38.965 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:51.118 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:38.970 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:51.123 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:38.985 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:38:51.138 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.018 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:51.172 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.019 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:51.116 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.021 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:51.117 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.023 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:51.177 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.033 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:38:51.129 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:39.036 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:38:51.189 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:48.961 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:01.114 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.010 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:01.163 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.032 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:01.186 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.059 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:01.212 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.059 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:01.156 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.062 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:01.158 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.063 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:01.216 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.084 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:01.180 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:49.088 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:01.240 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:06.244 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.139 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:06.236 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.140 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:06.237 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.145 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:06.298 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.155 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:06.251 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.159 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:06.312 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.187 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:06.341 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.193 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:06.346 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:11:54.205 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:06.358 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.065 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:21.218 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.069 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:21.222 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.089 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:21.242 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.122 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:21.276 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.123 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:21.219 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.124 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:21.277 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.124 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:21.221 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.139 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:21.236 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:09.141 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:21.294 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.081 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:26.234 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.085 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:26.239 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.098 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:26.251 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.135 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:26.288 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.135 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:26.231 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.137 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:26.233 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.138 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:26.291 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.148 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:26.244 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:14.154 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:26.305 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.074 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:41.227 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.125 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:41.278 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.143 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:41.296 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.175 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:41.329 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.178 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:41.274 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.180 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:41.277 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.181 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:41.334 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.196 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:41.292 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:29.197 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:41.351 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.137 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:46.290 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.142 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:46.296 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.154 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:39:46.308 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.192 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:46.346 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.194 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:46.348 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.198 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:46.291 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.198 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:46.293 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.208 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:39:46.305 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:34.209 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:39:46.363 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.175 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:01.328 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.179 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:01.332 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.197 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:01.350 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.233 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:01.386 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.233 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:01.329 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.234 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:01.388 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.235 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:01.331 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.251 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:01.347 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:49.252 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:01.405 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.196 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:06.350 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.201 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:06.354 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.214 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:06.367 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.250 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:06.403 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.252 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:06.349 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.253 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:06.406 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.255 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:06.352 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.264 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:06.418 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:12:54.267 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:06.363 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.429 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:12.581 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.430 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:12.583 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.451 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:12.604 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.486 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:12.582 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.488 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:12.584 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.491 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:12.644 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.494 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:12.647 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.503 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:12.600 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:00.508 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:12.661 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.069 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:13:09.067+0000 12a62640 -1 osd.6 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.390330+0000 front 2023-11-02T16:11:44.343336+0000 (oldest deadline 2023-11-02T16:13:08.869917+0000) 2023-11-02T16:13:09.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:13:09.202+0000 12a62640 -1 osd.0 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.254201+0000 front 2023-11-02T16:11:43.258909+0000 (oldest deadline 2023-11-02T16:13:09.150406+0000) 2023-11-02T16:13:09.237 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:21.390 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.242 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:21.396 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.261 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:21.414 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.294 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:21.390 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.296 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:21.449 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.296 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:21.392 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.298 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:21.452 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.308 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:21.404 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.310 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:21.464 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:09.560 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:13:09.557+0000 12a62640 -1 osd.5 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.247188+0000 front 2023-11-02T16:11:43.254079+0000 (oldest deadline 2023-11-02T16:13:09.142759+0000) 2023-11-02T16:13:09.595 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T16:13:09.593+0000 12a62640 -1 osd.4 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.938762+0000 front 2023-11-02T16:11:44.939345+0000 (oldest deadline 2023-11-02T16:13:09.031086+0000) 2023-11-02T16:13:10.063 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:13:10.061+0000 12a62640 -1 osd.6 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.390330+0000 front 2023-11-02T16:11:44.343336+0000 (oldest deadline 2023-11-02T16:13:08.869917+0000) 2023-11-02T16:13:10.165 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:13:10.163+0000 12a62640 -1 osd.0 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.254201+0000 front 2023-11-02T16:11:43.258909+0000 (oldest deadline 2023-11-02T16:13:09.150406+0000) 2023-11-02T16:13:10.574 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:13:10.571+0000 12a62640 -1 osd.5 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.247188+0000 front 2023-11-02T16:11:43.254079+0000 (oldest deadline 2023-11-02T16:13:09.142759+0000) 2023-11-02T16:13:10.581 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T16:13:10.579+0000 12a62640 -1 osd.4 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.938762+0000 front 2023-11-02T16:11:44.939345+0000 (oldest deadline 2023-11-02T16:13:09.031086+0000) 2023-11-02T16:13:11.103 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:13:11.101+0000 12a62640 -1 osd.6 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.390330+0000 front 2023-11-02T16:11:44.343336+0000 (oldest deadline 2023-11-02T16:13:08.869917+0000) 2023-11-02T16:13:11.143 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:13:11.140+0000 12a62640 -1 osd.0 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.254201+0000 front 2023-11-02T16:11:43.258909+0000 (oldest deadline 2023-11-02T16:13:09.150406+0000) 2023-11-02T16:13:11.552 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:13:11.549+0000 12a62640 -1 osd.5 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:43.247188+0000 front 2023-11-02T16:11:43.254079+0000 (oldest deadline 2023-11-02T16:13:09.142759+0000) 2023-11-02T16:13:11.594 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T16:13:11.591+0000 12a62640 -1 osd.4 1018 heartbeat_check: no reply from 172.21.15.174:6814 osd.2 since back 2023-11-02T16:11:44.938762+0000 front 2023-11-02T16:11:44.939345+0000 (oldest deadline 2023-11-02T16:13:09.031086+0000) 2023-11-02T16:13:14.329 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:26.482 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.391 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:26.543 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.405 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:26.558 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.441 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:26.538 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.442 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:26.539 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.444 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:26.597 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.446 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:26.599 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.456 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:26.553 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.463 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:26.616 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:14.924 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T16:13:14.921+0000 14265640 -1 osd.2 1021 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-02T16:13:22.419 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (8840 ms) 2023-11-02T16:13:22.420 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2023-11-02T16:13:22.420 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (14822 ms) 2023-11-02T16:13:22.420 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2023-11-02T16:13:22.420 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (9906 ms) 2023-11-02T16:13:22.420 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2023-11-02T16:13:22.421 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (16057 ms) 2023-11-02T16:13:22.421 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2023-11-02T16:13:22.421 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (10195 ms) 2023-11-02T16:13:22.421 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2023-11-02T16:13:22.421 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (10006 ms) 2023-11-02T16:13:22.422 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2023-11-02T16:13:22.422 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: hmm, no HitSet yet 2023-11-02T16:13:22.422 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: ok, hit_set contains 251:602f83fe:::foo:head 2023-11-02T16:13:22.422 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (10090 ms) 2023-11-02T16:13:22.422 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2023-11-02T16:13:22.423 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,0 2023-11-02T16:13:22.423 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first is 1698941354 2023-11-02T16:13:22.423 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,0 2023-11-02T16:13:22.423 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,0 2023-11-02T16:13:22.423 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,0 2023-11-02T16:13:22.424 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,0 2023-11-02T16:13:22.424 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,0 2023-11-02T16:13:22.424 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,0 2023-11-02T16:13:22.425 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,0 2023-11-02T16:13:22.425 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,0 2023-11-02T16:13:22.425 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,1698941362,1698941363,0 2023-11-02T16:13:22.425 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,1698941362,1698941363,0 2023-11-02T16:13:22.425 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941354,1698941356,1698941357,1698941359,1698941360,1698941362,1698941363,0 2023-11-02T16:13:22.426 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1698941357,1698941359,1698941360,1698941362,1698941363,1698941366,1698941367,0 2023-11-02T16:13:22.426 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first now 1698941357, trimmed 2023-11-02T16:13:22.426 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (22430 ms) 2023-11-02T16:13:22.426 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2023-11-02T16:13:22.426 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: foo0 2023-11-02T16:13:22.427 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: verifying foo0 is eventually promoted 2023-11-02T16:13:22.427 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (17649 ms) 2023-11-02T16:13:22.427 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2023-11-02T16:13:22.427 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (20324 ms) 2023-11-02T16:13:22.427 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2023-11-02T16:13:22.428 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (25111 ms) 2023-11-02T16:13:22.428 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2023-11-02T16:13:22.428 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (71459 ms) 2023-11-02T16:13:22.428 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2023-11-02T16:13:22.428 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3776 ms) 2023-11-02T16:13:22.429 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2023-11-02T16:13:22.429 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (88020 ms) 2023-11-02T16:13:22.429 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2023-11-02T16:13:22.429 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (4765 ms) 2023-11-02T16:13:22.429 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (648274 ms total) 2023-11-02T16:13:22.430 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2023-11-02T16:13:22.430 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] Global test environment tear-down 2023-11-02T16:13:22.430 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (2295836 ms total) 2023-11-02T16:13:22.430 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ PASSED ] 77 tests. 2023-11-02T16:13:22.435 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.435 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91343 2023-11-02T16:13:22.435 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91343 2023-11-02T16:13:22.435 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.435 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91593 2023-11-02T16:13:22.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91593 2023-11-02T16:13:22.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=92084 2023-11-02T16:13:22.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 92084 2023-11-02T16:13:22.436 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.437 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91765 2023-11-02T16:13:22.437 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91765 2023-11-02T16:13:22.437 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.437 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91704 2023-11-02T16:13:22.437 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91704 2023-11-02T16:13:22.438 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.438 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91420 2023-11-02T16:13:22.438 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91420 2023-11-02T16:13:22.438 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.438 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91741 2023-11-02T16:13:22.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91741 2023-11-02T16:13:22.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91442 2023-11-02T16:13:22.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91442 2023-11-02T16:13:22.439 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91918 2023-11-02T16:13:22.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91918 2023-11-02T16:13:22.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91318 2023-11-02T16:13:22.440 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91318 2023-11-02T16:13:22.441 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.441 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91383 2023-11-02T16:13:22.441 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91383 2023-11-02T16:13:22.441 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.441 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91959 2023-11-02T16:13:22.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91959 2023-11-02T16:13:22.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91309 2023-11-02T16:13:22.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91309 2023-11-02T16:13:22.442 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.443 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91329 2023-11-02T16:13:22.443 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91329 2023-11-02T16:13:22.443 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.443 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91627 2023-11-02T16:13:22.443 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91627 2023-11-02T16:13:22.444 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.444 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91727 2023-11-02T16:13:22.444 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91727 2023-11-02T16:13:22.444 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.444 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91667 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91667 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=92044 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 92044 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.445 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91908 2023-11-02T16:13:22.446 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91908 2023-11-02T16:13:22.446 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.446 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91520 2023-11-02T16:13:22.446 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91520 2023-11-02T16:13:22.446 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.447 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91784 2023-11-02T16:13:22.447 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91784 2023-11-02T16:13:22.447 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.447 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91302 2023-11-02T16:13:22.447 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91302 2023-11-02T16:13:22.448 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.448 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91362 2023-11-02T16:13:22.448 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91362 2023-11-02T16:13:22.448 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.448 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91756 2023-11-02T16:13:22.449 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91756 2023-11-02T16:13:22.449 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.449 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91398 2023-11-02T16:13:22.449 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91398 2023-11-02T16:13:22.449 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.450 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91817 2023-11-02T16:13:22.450 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91817 2023-11-02T16:13:22.450 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2023-11-02T16:13:22.450 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=91844 2023-11-02T16:13:22.450 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 91844 2023-11-02T16:13:22.451 INFO:tasks.workunit.client.0.smithi174.stderr:+ exit 0 2023-11-02T16:13:22.451 INFO:tasks.workunit.client.0.smithi174.stderr:+ cleanup 2023-11-02T16:13:22.451 INFO:tasks.workunit.client.0.smithi174.stderr:+ pkill -P 91297 2023-11-02T16:13:22.452 INFO:tasks.workunit.client.0.smithi174.stderr:+ true 2023-11-02T16:13:22.453 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-02T16:13:22.453 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-02T16:13:22.482 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2023-11-02T16:13:22.482 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-02T16:13:22.810 DEBUG:teuthology.parallel:result is None 2023-11-02T16:13:22.811 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-11-02T16:13:22.838 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-11-02T16:13:22.839 DEBUG:teuthology.orchestra.run.smithi174:> rmdir -- /home/ubuntu/cephtest/mnt.0 2023-11-02T16:13:22.903 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2023-11-02T16:13:22.904 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-02T16:13:22.916 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-02T16:13:22.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:13:23.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.040+0000 7fec0ae7a640 10 monclient: get_monmap_and_config 2023-11-02T16:13:23.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.040+0000 7fec0ae7a640 10 monclient: build_initial_monmap 2023-11-02T16:13:23.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.040+0000 7fec0ae7a640 10 monclient: monmap: 2023-11-02T16:13:23.043 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:23.043 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.043 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:23.041431+0000 2023-11-02T16:13:23.043 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:23.041431+0000 2023-11-02T16:13:23.044 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:23.044 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.044 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:23.044 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:23.044 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:23.045 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:23.045 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:23.045 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.041+0000 7fec0ae7a640 1 Processor -- start 2023-11-02T16:13:23.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.041+0000 7fec0ae7a640 1 -- start start 2023-11-02T16:13:23.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.041+0000 7fec0ae7a640 10 monclient: init 2023-11-02T16:13:23.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:23.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2023-11-02T16:13:23.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec040aa4a0 0x7fec040abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-b con 0x7fec040aa4a0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:23.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fec0406a790 0x7fec040aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-c con 0x7fec0406a790 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:23.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fec0406abf0 0x7fec0413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-a con 0x7fec0406abf0 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:23.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec040aa880 con 0x7fec0406abf0 2023-11-02T16:13:23.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec04071250 con 0x7fec0406a790 2023-11-02T16:13:23.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec040713c0 con 0x7fec040aa4a0 2023-11-02T16:13:23.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:23.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec0ae7a640 10 monclient(hunting): authenticate will time out at 3730.383766s 2023-11-02T16:13:23.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec03fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fec0406a790 0x7fec040aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec08c16640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec040aa4a0 0x7fec040abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.042+0000 7fec09417640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fec0406abf0 0x7fec0413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fec0406a790 0x7fec040aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:39446/0 (socket says 172.21.15.174:39446) 2023-11-02T16:13:23.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 1 -- 172.21.15.174:0/2976539160 learned_addr learned my addr 172.21.15.174:0/2976539160 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:23.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): get_auth_request con 0x7fec0406a790 auth_method 0 2023-11-02T16:13:23.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): get_auth_request con 0x7fec040aa4a0 auth_method 0 2023-11-02T16:13:23.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec09417640 10 monclient(hunting): get_auth_request con 0x7fec0406abf0 auth_method 0 2023-11-02T16:13:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec09417640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec09417640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec09417640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.043+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.044+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.044+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.044+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 10 monclient(hunting): handle_auth_done global_id 21481 payload 274 2023-11-02T16:13:23.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 -- 172.21.15.174:0/2976539160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec040aa4a0 msgr2=0x7fec040abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:23.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 --2- 172.21.15.174:0/2976539160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec040aa4a0 0x7fec040abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 -- 172.21.15.174:0/2976539160 >> v2:172.21.15.174:3300/0 conn(0x7fec0406abf0 msgr2=0x7fec0413cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:23.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 --2- 172.21.15.174:0/2976539160 >> v2:172.21.15.174:3300/0 conn(0x7fec0406abf0 0x7fec0413cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 10 monclient: _finish_hunting 0 2023-11-02T16:13:23.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 monclient: found mon.noname-c 2023-11-02T16:13:23.062 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:23.063 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:23.063 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.045+0000 7fec03fff640 1 -- 172.21.15.174:0/2976539160 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec040af150 con 0x7fec0406a790 2023-11-02T16:13:23.069 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.048+0000 7fec03fff640 1 --2- 172.21.15.174:0/2976539160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040aec10 secure :-1 s=READY pgs=5253 cs=0 l=1 rev1=1 crypto rx=0x7fec040722d0 tx=0x7febf002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b0cf30872470c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.070 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 1 -- 172.21.15.174:0/2976539160 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf00317c0 con 0x7fec0406a790 2023-11-02T16:13:23.070 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.070 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:23.070 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: dump: 2023-11-02T16:13:23.071 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.071 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.071 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.072 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.072 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.072 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.073 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.073 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.073 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.073 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:23.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:23.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:23.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 1 -- 172.21.15.174:0/2976539160 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf0031920 con 0x7fec0406a790 2023-11-02T16:13:23.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 5 monclient: authenticate success, global_id 21481 2023-11-02T16:13:23.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:23.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:23.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:23.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 4 set_mon_vals no callback set 2023-11-02T16:13:23.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 1 -- 172.21.15.174:0/2976539160 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0031cb0 con 0x7fec0406a790 2023-11-02T16:13:23.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:23.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec037fe640 10 monclient: dump: 2023-11-02T16:13:23.077 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.077 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.077 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.077 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.078 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.078 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.078 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.078 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.079 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.079 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 10 monclient: shutdown 2023-11-02T16:13:23.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:23.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 -- 172.21.15.174:0/2976539160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 msgr2=0x7fec040aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/2976539160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040aec10 secure :-1 s=READY pgs=5253 cs=0 l=1 rev1=1 crypto rx=0x7fec040722d0 tx=0x7febf002b860 comp rx=0 tx=0).stop 2023-11-02T16:13:23.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 -- 172.21.15.174:0/2976539160 shutdown_connections 2023-11-02T16:13:23.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/2976539160 >> v2:172.21.15.174:3300/0 conn(0x7fec0406abf0 0x7fec0413cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/2976539160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040aec10 unknown :-1 s=CLOSED pgs=5253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/2976539160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec040aa4a0 0x7fec040abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 -- 172.21.15.174:0/2976539160 >> 172.21.15.174:0/2976539160 conn(0x7fec04069c70 msgr2=0x7fec0406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:23.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 -- 172.21.15.174:0/2976539160 shutdown_connections 2023-11-02T16:13:23.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.052+0000 7fec0ae7a640 1 -- 172.21.15.174:0/2976539160 wait complete. 2023-11-02T16:13:23.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.053+0000 7fec0ae7a640 10 monclient: build_initial_monmap 2023-11-02T16:13:23.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.053+0000 7fec0ae7a640 10 monclient: monmap: 2023-11-02T16:13:23.082 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:23.082 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:23.082 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:23.083 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:23.083 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:23.083 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.084 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:23.084 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:23.084 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:23.084 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.053+0000 7fec0ae7a640 1 librados: starting msgr at 2023-11-02T16:13:23.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.053+0000 7fec0ae7a640 1 librados: starting objecter 2023-11-02T16:13:23.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 Processor -- start 2023-11-02T16:13:23.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 -- start start 2023-11-02T16:13:23.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 librados: setting wanted keys 2023-11-02T16:13:23.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 librados: calling monclient init 2023-11-02T16:13:23.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient: init 2023-11-02T16:13:23.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:23.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:13:23.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-b con 0x7fec0406a790 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:23.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec0406abf0 0x7fec04070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-c con 0x7fec0406abf0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:23.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 0x7fec041d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): picked mon.noname-a con 0x7fec040aa4a0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:23.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec04070a90 con 0x7fec040aa4a0 2023-11-02T16:13:23.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040707c0 con 0x7fec0406a790 2023-11-02T16:13:23.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec041446b0 con 0x7fec0406abf0 2023-11-02T16:13:23.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:23.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.054+0000 7fec0ae7a640 10 monclient(hunting): authenticate will time out at 3730.396484s 2023-11-02T16:13:23.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 0x7fec041d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec08c16640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec03fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec0406abf0 0x7fec04070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 0x7fec041d9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44492/0 (socket says 172.21.15.174:44492) 2023-11-02T16:13:23.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 1 -- 172.21.15.174:0/1993398276 learned_addr learned my addr 172.21.15.174:0/1993398276 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:23.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 10 monclient(hunting): get_auth_request con 0x7fec040aa4a0 auth_method 0 2023-11-02T16:13:23.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec09417640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec08c16640 10 monclient(hunting): get_auth_request con 0x7fec0406a790 auth_method 0 2023-11-02T16:13:23.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec08c16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec08c16640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec08c16640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec03fff640 10 monclient(hunting): get_auth_request con 0x7fec0406abf0 auth_method 0 2023-11-02T16:13:23.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec03fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.055+0000 7fec03fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec08c16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec09417640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.056+0000 7fec03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.058+0000 7fec08c16640 10 monclient(hunting): handle_auth_done global_id 21484 payload 824 2023-11-02T16:13:23.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.058+0000 7fec08c16640 1 -- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec0406abf0 msgr2=0x7fec04070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.058+0000 7fec08c16640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec0406abf0 0x7fec04070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 1 -- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 msgr2=0x7fec041d9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 0x7fec041d9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 10 monclient: _finish_hunting 0 2023-11-02T16:13:23.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 1 monclient: found mon.noname-b 2023-11-02T16:13:23.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:23.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:23.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.059+0000 7fec08c16640 1 -- 172.21.15.174:0/1993398276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00094d0 con 0x7fec0406a790 2023-11-02T16:13:23.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.061+0000 7fec08c16640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040b2230 secure :-1 s=READY pgs=5254 cs=0 l=1 rev1=1 crypto rx=0x7febf800e8a0 tx=0x7febf800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.061+0000 7fec017fa640 10 client.?.objecter ms_handle_connect 0x7fec0406a790 2023-11-02T16:13:23.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.061+0000 7fec017fa640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:23.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8009e60 con 0x7fec0406a790 2023-11-02T16:13:23.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: dump: 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.100 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.101 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.101 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.101 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.101 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.101 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: _finish_auth 0 2023-11-02T16:13:23.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: _check_auth_tickets 2023-11-02T16:13:23.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:23.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf800c940 con 0x7fec0406a790 2023-11-02T16:13:23.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 5 monclient: authenticate success, global_id 21484 2023-11-02T16:13:23.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 10 monclient: _renew_subs 2023-11-02T16:13:23.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:23.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0406eef0 con 0x7fec0406a790 2023-11-02T16:13:23.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:23.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800ccd0 con 0x7fec0406a790 2023-11-02T16:13:23.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:23.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec017fa640 10 monclient: dump: 2023-11-02T16:13:23.104 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.104 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.105 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.106 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.106 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec02ffd640 4 set_mon_vals no callback set 2023-11-02T16:13:23.106 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 10 client.21484.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:23.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 10 monclient: _renew_subs 2023-11-02T16:13:23.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:23.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec041da4f0 con 0x7fec0406a790 2023-11-02T16:13:23.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.064+0000 7fec0ae7a640 1 librados: init done 2023-11-02T16:13:23.107 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.065+0000 7fec0ae7a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.065+0000 7fec0ae7a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.065+0000 7fec0ae7a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:23.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.065+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0404e040 con 0x7fec0406a790 2023-11-02T16:13:23.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.309+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7febf8013470 con 0x7fec0406a790 2023-11-02T16:13:23.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.309+0000 7fec017fa640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 0x7febe806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.310+0000 7fec03fff640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 0x7febe806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.310+0000 7fec03fff640 10 monclient: get_auth_request con 0x7febe806be10 auth_method 0 2023-11-02T16:13:23.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.310+0000 7fec03fff640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 0x7febe806e2d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febf0036000 tx=0x7febf003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.310+0000 7fec017fa640 10 client.21484.objecter ms_handle_connect 0x7febe806be10 2023-11-02T16:13:23.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.312+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1029..1029 src has 247..1029) v4 ==== 6282+0+0 (secure 0 0 0) 0x7febf8058370 con 0x7fec0406a790 2023-11-02T16:13:23.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.312+0000 7fec017fa640 10 client.21484.objecter ms_dispatch 0x7fec04000b90 osd_map(1029..1029 src has 247..1029) v4 2023-11-02T16:13:23.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.312+0000 7fec017fa640 3 client.21484.objecter handle_osd_map got epochs [1029,1029] > 0 2023-11-02T16:13:23.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.312+0000 7fec017fa640 3 client.21484.objecter handle_osd_map decoding full epoch 1029 2023-11-02T16:13:23.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.312+0000 7fec017fa640 20 client.21484.objecter dump_active .. 0 homeless 2023-11-02T16:13:23.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.322+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7febf80546b0 con 0x7fec0406a790 2023-11-02T16:13:23.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.322+0000 7fec017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.322+0000 7fec017fa640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:23.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.496+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec040af470 con 0x7febe806be10 2023-11-02T16:13:23.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.500+0000 7fec017fa640 1 -- 172.21.15.174:0/1993398276 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152018 (secure 0 0 0) 0x7fec040af470 con 0x7febe806be10 2023-11-02T16:13:23.504 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:23.506 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:13:23.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 10 librados: watch_flush enter 2023-11-02T16:13:23.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 10 librados: watch_flush exit 2023-11-02T16:13:23.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 20 client.21484.objecter shutdown clearing up homeless session... 2023-11-02T16:13:23.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 10 client.21484.objecter successfully canceled tick 2023-11-02T16:13:23.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 msgr2=0x7febe806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 0x7febe806e2d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febf0036000 tx=0x7febf003d040 comp rx=0 tx=0).stop 2023-11-02T16:13:23.510 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 10 monclient: shutdown 2023-11-02T16:13:23.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:23.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 msgr2=0x7fec040b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040b2230 secure :-1 s=READY pgs=5254 cs=0 l=1 rev1=1 crypto rx=0x7febf800e8a0 tx=0x7febf800ed60 comp rx=0 tx=0).stop 2023-11-02T16:13:23.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 shutdown_connections 2023-11-02T16:13:23.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec040aa4a0 0x7fec041d9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7febe806be10 0x7febe806e2d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec0406abf0 0x7fec04070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 --2- 172.21.15.174:0/1993398276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0406a790 0x7fec040b2230 unknown :-1 s=CLOSED pgs=5254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 >> 172.21.15.174:0/1993398276 conn(0x7fec04069c70 msgr2=0x7fec0406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:23.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 shutdown_connections 2023-11-02T16:13:23.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 -- 172.21.15.174:0/1993398276 wait complete. 2023-11-02T16:13:23.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.503+0000 7fec0ae7a640 1 librados: shutdown 2023-11-02T16:13:23.515 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2218,"stamp":"2023-11-02T16:13:23.403888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609908,"kb_used_data":86068,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364812,"statfs":{"total":773094113280,"available":772469567488,"internally_reserved":0,"allocated":88133632,"data_stored":51603970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":432,"apply_latency_ms":432,"commit_latency_ns":432000000,"apply_latency_ns":432000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.400285"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":998,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.956626+0000","last_change":"2023-11-02T16:13:17.157671+0000","last_active":"2023-11-02T16:13:17.956626+0000","last_peered":"2023-11-02T16:13:17.956626+0000","last_clean":"2023-11-02T16:13:17.956626+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:17.956626+0000","last_undegraded":"2023-11-02T16:13:17.956626+0000","last_fullsized":"2023-11-02T16:13:17.956626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:58:17.271177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.749561+0000","last_change":"2023-11-02T16:07:27.307241+0000","last_active":"2023-11-02T16:13:17.749561+0000","last_peered":"2023-11-02T16:13:17.749561+0000","last_clean":"2023-11-02T16:13:17.749561+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:17.749561+0000","last_undegraded":"2023-11-02T16:13:17.749561+0000","last_fullsized":"2023-11-02T16:13:17.749561+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:12:49.486828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.186771+0000","last_change":"2023-11-02T16:07:27.395825+0000","last_active":"2023-11-02T16:13:21.186771+0000","last_peered":"2023-11-02T16:13:21.186771+0000","last_clean":"2023-11-02T16:13:21.186771+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:21.186771+0000","last_undegraded":"2023-11-02T16:13:21.186771+0000","last_fullsized":"2023-11-02T16:13:21.186771+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:25:43.494367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":972,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.121930+0000","last_change":"2023-11-02T15:35:17.345372+0000","last_active":"2023-11-02T16:13:19.121930+0000","last_peered":"2023-11-02T16:13:19.121930+0000","last_clean":"2023-11-02T16:13:19.121930+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:19.121930+0000","last_undegraded":"2023-11-02T16:13:19.121930+0000","last_fullsized":"2023-11-02T16:13:19.121930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:39:06.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.159925+0000","last_change":"2023-11-02T15:35:14.211126+0000","last_active":"2023-11-02T16:13:21.159925+0000","last_peered":"2023-11-02T16:13:21.159925+0000","last_clean":"2023-11-02T16:13:21.159925+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:21.159925+0000","last_undegraded":"2023-11-02T16:13:21.159925+0000","last_fullsized":"2023-11-02T16:13:21.159925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:19:16.985396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":920,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.171904+0000","last_change":"2023-11-02T15:59:17.698985+0000","last_active":"2023-11-02T16:13:21.171904+0000","last_peered":"2023-11-02T16:13:21.171904+0000","last_clean":"2023-11-02T16:13:21.171904+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:21.171904+0000","last_undegraded":"2023-11-02T16:13:21.171904+0000","last_fullsized":"2023-11-02T16:13:21.171904+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:31:01.071757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":998,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.146603+0000","last_change":"2023-11-02T16:13:16.820908+0000","last_active":"2023-11-02T16:13:21.146603+0000","last_peered":"2023-11-02T16:13:21.146603+0000","last_clean":"2023-11-02T16:13:21.146603+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:21.146603+0000","last_undegraded":"2023-11-02T16:13:21.146603+0000","last_fullsized":"2023-11-02T16:13:21.146603+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:08:51.272032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.143826+0000","last_change":"2023-11-02T15:35:30.105402+0000","last_active":"2023-11-02T16:13:21.143826+0000","last_peered":"2023-11-02T16:13:21.143826+0000","last_clean":"2023-11-02T16:13:21.143826+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:21.143826+0000","last_undegraded":"2023-11-02T16:13:21.143826+0000","last_fullsized":"2023-11-02T16:13:21.143826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:00:16.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":960,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.061767+0000","last_change":"2023-11-02T16:07:27.407922+0000","last_active":"2023-11-02T16:13:20.061767+0000","last_peered":"2023-11-02T16:13:20.061767+0000","last_clean":"2023-11-02T16:13:20.061767+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:20.061767+0000","last_undegraded":"2023-11-02T16:13:20.061767+0000","last_fullsized":"2023-11-02T16:13:20.061767+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:16:50.921541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1001,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.087417+0000","last_change":"2023-11-02T16:13:16.885325+0000","last_active":"2023-11-02T16:13:20.087417+0000","last_peered":"2023-11-02T16:13:20.087417+0000","last_clean":"2023-11-02T16:13:20.087417+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:20.087417+0000","last_undegraded":"2023-11-02T16:13:20.087417+0000","last_fullsized":"2023-11-02T16:13:20.087417+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:18:37.960941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1011,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.069466+0000","last_change":"2023-11-02T16:13:16.913467+0000","last_active":"2023-11-02T16:13:20.069466+0000","last_peered":"2023-11-02T16:13:20.069466+0000","last_clean":"2023-11-02T16:13:20.069466+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:20.069466+0000","last_undegraded":"2023-11-02T16:13:20.069466+0000","last_fullsized":"2023-11-02T16:13:20.069466+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:05:41.639058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.719637+0000","last_change":"2023-11-02T16:13:17.189587+0000","last_active":"2023-11-02T16:13:17.719637+0000","last_peered":"2023-11-02T16:13:17.719637+0000","last_clean":"2023-11-02T16:13:17.719637+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:17.719637+0000","last_undegraded":"2023-11-02T16:13:17.719637+0000","last_fullsized":"2023-11-02T16:13:17.719637+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:04:14.846503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":983,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.746564+0000","last_change":"2023-11-02T15:59:17.640186+0000","last_active":"2023-11-02T16:13:17.746564+0000","last_peered":"2023-11-02T16:13:17.746564+0000","last_clean":"2023-11-02T16:13:17.746564+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:17.746564+0000","last_undegraded":"2023-11-02T16:13:17.746564+0000","last_fullsized":"2023-11-02T16:13:17.746564+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:57.212864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.744965+0000","last_change":"2023-11-02T15:59:17.577430+0000","last_active":"2023-11-02T16:13:17.744965+0000","last_peered":"2023-11-02T16:13:17.744965+0000","last_clean":"2023-11-02T16:13:17.744965+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:17.744965+0000","last_undegraded":"2023-11-02T16:13:17.744965+0000","last_fullsized":"2023-11-02T16:13:17.744965+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:03:26.273290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":973,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.959279+0000","last_change":"2023-11-02T15:35:23.837202+0000","last_active":"2023-11-02T16:13:17.959279+0000","last_peered":"2023-11-02T16:13:17.959279+0000","last_clean":"2023-11-02T16:13:17.959279+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:17.959279+0000","last_undegraded":"2023-11-02T16:13:17.959279+0000","last_fullsized":"2023-11-02T16:13:17.959279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:24.083855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":978,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.094447+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T16:13:19.094447+0000","last_peered":"2023-11-02T16:13:19.094447+0000","last_clean":"2023-11-02T16:13:19.094447+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:19.094447+0000","last_undegraded":"2023-11-02T16:13:19.094447+0000","last_fullsized":"2023-11-02T16:13:19.094447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.954242+0000","last_change":"2023-11-02T15:59:17.883690+0000","last_active":"2023-11-02T16:13:17.954242+0000","last_peered":"2023-11-02T16:13:17.954242+0000","last_clean":"2023-11-02T16:13:17.954242+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:17.954242+0000","last_undegraded":"2023-11-02T16:13:17.954242+0000","last_fullsized":"2023-11-02T16:13:17.954242+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:58:33.336681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":989,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.718156+0000","last_change":"2023-11-02T15:59:17.612075+0000","last_active":"2023-11-02T16:13:17.718156+0000","last_peered":"2023-11-02T16:13:17.718156+0000","last_clean":"2023-11-02T16:13:17.718156+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:17.718156+0000","last_undegraded":"2023-11-02T16:13:17.718156+0000","last_fullsized":"2023-11-02T16:13:17.718156+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:08:04.600098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1013,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.071639+0000","last_change":"2023-11-02T16:13:17.151154+0000","last_active":"2023-11-02T16:13:20.071639+0000","last_peered":"2023-11-02T16:13:20.071639+0000","last_clean":"2023-11-02T16:13:20.071639+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:20.071639+0000","last_undegraded":"2023-11-02T16:13:20.071639+0000","last_fullsized":"2023-11-02T16:13:20.071639+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:40.667953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":952,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.080989+0000","last_change":"2023-11-02T15:43:56.978148+0000","last_active":"2023-11-02T16:13:20.080989+0000","last_peered":"2023-11-02T16:13:20.080989+0000","last_clean":"2023-11-02T16:13:20.080989+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:20.080989+0000","last_undegraded":"2023-11-02T16:13:20.080989+0000","last_fullsized":"2023-11-02T16:13:20.080989+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:17:20.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":530,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.732985+0000","last_change":"2023-11-02T16:07:27.487367+0000","last_active":"2023-11-02T16:13:17.732985+0000","last_peered":"2023-11-02T16:13:17.732985+0000","last_clean":"2023-11-02T16:13:17.732985+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:17.732985+0000","last_undegraded":"2023-11-02T16:13:17.732985+0000","last_fullsized":"2023-11-02T16:13:17.732985+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:15:14.310785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1004,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.696771+0000","last_change":"2023-11-02T15:59:17.535951+0000","last_active":"2023-11-02T16:13:17.696771+0000","last_peered":"2023-11-02T16:13:17.696771+0000","last_clean":"2023-11-02T16:13:17.696771+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:17.696771+0000","last_undegraded":"2023-11-02T16:13:17.696771+0000","last_fullsized":"2023-11-02T16:13:17.696771+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:50:51.454223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":975,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.934494+0000","last_change":"2023-11-02T15:35:16.434956+0000","last_active":"2023-11-02T16:13:17.934494+0000","last_peered":"2023-11-02T16:13:17.934494+0000","last_clean":"2023-11-02T16:13:17.934494+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:17.934494+0000","last_undegraded":"2023-11-02T16:13:17.934494+0000","last_fullsized":"2023-11-02T16:13:17.934494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:54:41.093040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":986,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.729224+0000","last_change":"2023-11-02T16:07:27.307462+0000","last_active":"2023-11-02T16:13:17.729224+0000","last_peered":"2023-11-02T16:13:17.729224+0000","last_clean":"2023-11-02T16:13:17.729224+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:17.729224+0000","last_undegraded":"2023-11-02T16:13:17.729224+0000","last_fullsized":"2023-11-02T16:13:17.729224+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:53:09.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":935,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.988390+0000","last_change":"2023-11-02T15:35:10.705051+0000","last_active":"2023-11-02T16:13:19.988390+0000","last_peered":"2023-11-02T16:13:19.988390+0000","last_clean":"2023-11-02T16:13:19.988390+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:19.988390+0000","last_undegraded":"2023-11-02T16:13:19.988390+0000","last_fullsized":"2023-11-02T16:13:19.988390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:36:19.002698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":990,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.703806+0000","last_change":"2023-11-02T15:59:17.407544+0000","last_active":"2023-11-02T16:13:17.703806+0000","last_peered":"2023-11-02T16:13:17.703806+0000","last_clean":"2023-11-02T16:13:17.703806+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:17.703806+0000","last_undegraded":"2023-11-02T16:13:17.703806+0000","last_fullsized":"2023-11-02T16:13:17.703806+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:52:17.493086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":973,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.156486+0000","last_change":"2023-11-02T15:35:30.088014+0000","last_active":"2023-11-02T16:13:21.156486+0000","last_peered":"2023-11-02T16:13:21.156486+0000","last_clean":"2023-11-02T16:13:21.156486+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:21.156486+0000","last_undegraded":"2023-11-02T16:13:21.156486+0000","last_fullsized":"2023-11-02T16:13:21.156486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:24:59.243202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.710169+0000","last_change":"2023-11-02T16:13:17.216261+0000","last_active":"2023-11-02T16:13:17.710169+0000","last_peered":"2023-11-02T16:13:17.710169+0000","last_clean":"2023-11-02T16:13:17.710169+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:17.710169+0000","last_undegraded":"2023-11-02T16:13:17.710169+0000","last_fullsized":"2023-11-02T16:13:17.710169+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:51:41.117006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"22'76","reported_seq":1085,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.149378+0000","last_change":"2023-11-02T15:33:55.693910+0000","last_active":"2023-11-02T16:13:21.149378+0000","last_peered":"2023-11-02T16:13:21.149378+0000","last_clean":"2023-11-02T16:13:21.149378+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:21.149378+0000","last_undegraded":"2023-11-02T16:13:21.149378+0000","last_fullsized":"2023-11-02T16:13:21.149378+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_clean_scrub_stamp":"2023-11-02T15:33:53.161912+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:29.329385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"40'7","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.936228+0000","last_change":"2023-11-02T16:13:17.029849+0000","last_active":"2023-11-02T16:13:17.936228+0000","last_peered":"2023-11-02T16:13:17.936228+0000","last_clean":"2023-11-02T16:13:17.936228+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:17.936228+0000","last_undegraded":"2023-11-02T16:13:17.936228+0000","last_fullsized":"2023-11-02T16:13:17.936228+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:45:56.714871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1002,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.038464+0000","last_change":"2023-11-02T16:13:17.007381+0000","last_active":"2023-11-02T16:13:20.038464+0000","last_peered":"2023-11-02T16:13:20.038464+0000","last_clean":"2023-11-02T16:13:20.038464+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:20.038464+0000","last_undegraded":"2023-11-02T16:13:20.038464+0000","last_fullsized":"2023-11-02T16:13:20.038464+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:29:31.495127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1002,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.079106+0000","last_change":"2023-11-02T16:13:16.951026+0000","last_active":"2023-11-02T16:13:19.079106+0000","last_peered":"2023-11-02T16:13:19.079106+0000","last_clean":"2023-11-02T16:13:19.079106+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:19.079106+0000","last_undegraded":"2023-11-02T16:13:19.079106+0000","last_fullsized":"2023-11-02T16:13:19.079106+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:01:56.841900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":962,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.028826+0000","last_change":"2023-11-02T16:13:17.159877+0000","last_active":"2023-11-02T16:13:20.028826+0000","last_peered":"2023-11-02T16:13:20.028826+0000","last_clean":"2023-11-02T16:13:20.028826+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:20.028826+0000","last_undegraded":"2023-11-02T16:13:20.028826+0000","last_fullsized":"2023-11-02T16:13:20.028826+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:32:03.253445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":987,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.120389+0000","last_change":"2023-11-02T15:43:57.129812+0000","last_active":"2023-11-02T16:13:19.120389+0000","last_peered":"2023-11-02T16:13:19.120389+0000","last_clean":"2023-11-02T16:13:19.120389+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:19.120389+0000","last_undegraded":"2023-11-02T16:13:19.120389+0000","last_fullsized":"2023-11-02T16:13:19.120389+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:42.294660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":976,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.092801+0000","last_change":"2023-11-02T15:35:17.386786+0000","last_active":"2023-11-02T16:13:19.092801+0000","last_peered":"2023-11-02T16:13:19.092801+0000","last_clean":"2023-11-02T16:13:19.092801+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:19.092801+0000","last_undegraded":"2023-11-02T16:13:19.092801+0000","last_fullsized":"2023-11-02T16:13:19.092801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:58:39.849736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":984,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.748036+0000","last_change":"2023-11-02T15:59:17.643298+0000","last_active":"2023-11-02T16:13:17.748036+0000","last_peered":"2023-11-02T16:13:17.748036+0000","last_clean":"2023-11-02T16:13:17.748036+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:17.748036+0000","last_undegraded":"2023-11-02T16:13:17.748036+0000","last_fullsized":"2023-11-02T16:13:17.748036+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:31:03.424299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":985,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.725844+0000","last_change":"2023-11-02T15:59:17.471575+0000","last_active":"2023-11-02T16:13:17.725844+0000","last_peered":"2023-11-02T16:13:17.725844+0000","last_clean":"2023-11-02T16:13:17.725844+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:17.725844+0000","last_undegraded":"2023-11-02T16:13:17.725844+0000","last_fullsized":"2023-11-02T16:13:17.725844+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:21.803164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.183490+0000","last_change":"2023-11-02T16:13:16.821060+0000","last_active":"2023-11-02T16:13:21.183490+0000","last_peered":"2023-11-02T16:13:21.183490+0000","last_clean":"2023-11-02T16:13:21.183490+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:21.183490+0000","last_undegraded":"2023-11-02T16:13:21.183490+0000","last_fullsized":"2023-11-02T16:13:21.183490+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:46.475704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.055317+0000","last_change":"2023-11-02T16:07:27.399974+0000","last_active":"2023-11-02T16:13:19.055317+0000","last_peered":"2023-11-02T16:13:19.055317+0000","last_clean":"2023-11-02T16:13:19.055317+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:19.055317+0000","last_undegraded":"2023-11-02T16:13:19.055317+0000","last_fullsized":"2023-11-02T16:13:19.055317+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:02:28.668857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.077626+0000","last_change":"2023-11-02T16:13:16.955603+0000","last_active":"2023-11-02T16:13:19.077626+0000","last_peered":"2023-11-02T16:13:19.077626+0000","last_clean":"2023-11-02T16:13:19.077626+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:19.077626+0000","last_undegraded":"2023-11-02T16:13:19.077626+0000","last_fullsized":"2023-11-02T16:13:19.077626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:12:56.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":997,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.158342+0000","last_change":"2023-11-02T16:13:16.834330+0000","last_active":"2023-11-02T16:13:21.158342+0000","last_peered":"2023-11-02T16:13:21.158342+0000","last_clean":"2023-11-02T16:13:21.158342+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:21.158342+0000","last_undegraded":"2023-11-02T16:13:21.158342+0000","last_fullsized":"2023-11-02T16:13:21.158342+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:56:31.478240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544251,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63832,"kb_used_data":10640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308008,"statfs":{"total":96636764160,"available":96571400192,"internally_reserved":0,"allocated":10895360,"data_stored":6318201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":15.131},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":6.7720000000000002}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":9.5830000000000002},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":13.759}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":10.568},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":8.0370000000000008}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":16.782},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":18.870000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":11.778},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":14.539999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":15.752000000000001},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":11.177}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":18.274999999999999},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":12.448}]}]},{"osd":1,"up_from":16,"seq":68719477187,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104348,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267492,"statfs":{"total":96636764160,"available":96529911808,"internally_reserved":0,"allocated":11292672,"data_stored":6716036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":8.7509999999999994,"5min":11.885,"15min":17.582000000000001},"min":{"1min":4.5650000000000004,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":13.523999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":13.021000000000001},{"interface":"front","average":{"1min":8.1389999999999993,"5min":12.238,"15min":18.489999999999998},"min":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":11.058999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":13.617000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":28.122},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":27.352}]},{"osd":3,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":11.907,"5min":13.795999999999999,"15min":21.059999999999999},"min":{"1min":8.5370000000000008,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":26.161000000000001,"5min":77.778000000000006,"15min":549.92200000000003},"last":11.098000000000001},{"interface":"front","average":{"1min":12.789999999999999,"5min":17.585999999999999,"15min":24.571000000000002},"min":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":7.8280000000000003},"max":{"1min":26.827000000000002,"5min":108.244,"15min":542.60500000000002},"last":7.3369999999999997}]},{"osd":4,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":107.46299999999999,"5min":110.161,"15min":49.548000000000002},"min":{"1min":7.8819999999999997,"5min":7.8819999999999997,"15min":6.1360000000000001},"max":{"1min":1884.9880000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":9.391},{"interface":"front","average":{"1min":111.589,"5min":110.36199999999999,"15min":51.369},"min":{"1min":7.6950000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"max":{"1min":1970.0229999999999,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":8.8599999999999994}]},{"osd":5,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":16.436,"15min":25.068999999999999},"min":{"1min":8.0990000000000002,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":16.541,"5min":71.225999999999999,"15min":529.93499999999995},"last":36.536999999999999},{"interface":"front","average":{"1min":15.509,"5min":19.974,"15min":26.859000000000002},"min":{"1min":9.5839999999999996,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":23.559999999999999,"5min":122.14,"15min":555.77700000000004},"last":29.494}]},{"osd":6,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":15.324999999999999,"5min":22.655000000000001,"15min":1553.2819999999999},"min":{"1min":9.0719999999999992,"5min":8.6210000000000004,"15min":8.1400000000000006},"max":{"1min":31.744,"5min":245.36500000000001,"15min":73455.679000000004},"last":14.265000000000001},{"interface":"front","average":{"1min":15.226000000000001,"5min":17.460999999999999,"15min":1549.6890000000001},"min":{"1min":9.5269999999999992,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":30.879999999999999,"5min":124.36499999999999,"15min":73453.190000000002},"last":11.747999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:12:35 2023","interfaces":[{"interface":"back","average":{"1min":15.096,"5min":18.516999999999999,"15min":21.047000000000001},"min":{"1min":10.757999999999999,"5min":10.097,"15min":10.097},"max":{"1min":21.617999999999999,"5min":91.019999999999996,"15min":91.019999999999996},"last":9.9969999999999999},{"interface":"front","average":{"1min":16.209,"5min":22.247,"15min":24.065999999999999},"min":{"1min":10.824999999999999,"5min":10.113,"15min":10.113},"max":{"1min":28.861999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":15.321999999999999}]}]},{"osd":0,"up_from":16,"seq":68719477211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81968,"kb_used_data":10728,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289872,"statfs":{"total":96636764160,"available":96552828928,"internally_reserved":0,"allocated":10985472,"data_stored":6426109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:12:55 2023","interfaces":[{"interface":"back","average":{"1min":8.7309999999999999,"5min":13.488,"15min":77.375},"min":{"1min":5.3490000000000002,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.488,"5min":107.59999999999999,"15min":5781.2129999999997},"last":7.5629999999999997},{"interface":"front","average":{"1min":8.4719999999999995,"5min":11.718,"15min":74.570999999999998},"min":{"1min":6.2249999999999996,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":13.672000000000001,"5min":73.094999999999999,"15min":5443.4889999999996},"last":11.455}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.778},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":13.651999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":12.1,"5min":17.452999999999999,"15min":17.625},"min":{"1min":8.3399999999999999,"5min":7.46,"15min":7.46},"max":{"1min":29.192,"5min":156.99100000000001,"15min":234.29300000000001},"last":40.524999999999999},{"interface":"front","average":{"1min":12.449999999999999,"5min":17.09,"15min":15.265000000000001},"min":{"1min":7.6239999999999997,"5min":7.4950000000000001,"15min":6.6070000000000002},"max":{"1min":34.267000000000003,"5min":151.30099999999999,"15min":151.30099999999999},"last":62.927}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":21213.947,"5min":17485.800999999999,"15min":5840.5320000000002},"min":{"1min":9.7330000000000005,"5min":7.1600000000000001,"15min":6.3200000000000003},"max":{"1min":64456.964,"5min":66161.077000000005,"15min":66161.077000000005},"last":14.664},{"interface":"front","average":{"1min":21222.184000000001,"5min":17490.220000000001,"15min":5843.1549999999997},"min":{"1min":11.762,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":64460.497000000003,"5min":66163.926000000007,"15min":66163.926000000007},"last":15.914}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":16.919},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":17.655000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":18.556999999999999,"5min":22.667000000000002,"15min":1772.4739999999999},"min":{"1min":14.353999999999999,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":38.277000000000001,"5min":107.51000000000001,"15min":71055.725999999995},"last":18.425000000000001},{"interface":"front","average":{"1min":13.893000000000001,"5min":19.393000000000001,"15min":1770.963},"min":{"1min":7.7320000000000002,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":32.561,"5min":156.91,"15min":71055.100000000006},"last":15.398}]},{"osd":7,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19.140000000000001,"5min":25.041,"15min":30.241},"min":{"1min":13.130000000000001,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":44.636000000000003,"5min":205.88999999999999,"15min":205.88999999999999},"last":19.449000000000002},{"interface":"front","average":{"1min":11.23,"5min":17.503,"15min":22.859000000000002},"min":{"1min":8.0190000000000001,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":31.449999999999999,"5min":155.96600000000001,"15min":155.96600000000001},"last":12.237}]}]},{"osd":6,"up_from":387,"seq":1662152343954,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63828,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308012,"statfs":{"total":96636764160,"available":96571404288,"internally_reserved":0,"allocated":10891264,"data_stored":6318196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.405000000000001,"5min":14.585000000000001,"15min":20.376999999999999},"min":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.218},"max":{"1min":208.60499999999999,"5min":208.60499999999999,"15min":572.92100000000005},"last":6.9589999999999996},{"interface":"front","average":{"1min":19.026,"5min":14.109999999999999,"15min":21.210000000000001},"min":{"1min":6.8200000000000003,"5min":4.6980000000000004,"15min":4.0419999999999998},"max":{"1min":172.51499999999999,"5min":172.51499999999999,"15min":630.37900000000002},"last":19.815000000000001}]},{"osd":1,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.109000000000002,"5min":16.506,"15min":1146.9380000000001},"min":{"1min":7.5700000000000003,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":77.540000000000006,"5min":143.55699999999999,"15min":41558.328999999998},"last":56.890000000000001},{"interface":"front","average":{"1min":21.494,"5min":17.533999999999999,"15min":1148.0730000000001},"min":{"1min":7.633,"5min":6.8019999999999996,"15min":5.8220000000000001},"max":{"1min":176.839,"5min":176.839,"15min":41560.286999999997},"last":20.495999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":9.1359999999999992},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":56.274999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":25.748000000000001,"5min":22.646000000000001,"15min":24.829000000000001},"min":{"1min":9.4220000000000006,"5min":8.5069999999999997,"15min":6.4480000000000004},"max":{"1min":260.11700000000002,"5min":260.11700000000002,"15min":633.899},"last":55.115000000000002},{"interface":"front","average":{"1min":20.872,"5min":19.306999999999999,"15min":25.396999999999998},"min":{"1min":8.0060000000000002,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":177.78100000000001,"5min":188.101,"15min":582.32399999999996},"last":8.0109999999999992}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17750.601999999999,"5min":4460.9930000000004,"15min":1504.991},"min":{"1min":10.298,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":66604.820000000007,"5min":67109.214999999997,"15min":67109.214999999997},"last":21.734000000000002},{"interface":"front","average":{"1min":17580.571,"5min":4423.1670000000004,"15min":1494.211},"min":{"1min":8.0969999999999995,"5min":8.0969999999999995,"15min":8.0969999999999995},"max":{"1min":66274.171000000002,"5min":66779.236000000004,"15min":66779.236000000004},"last":9.875}]},{"osd":5,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":27.411999999999999,"5min":22.401,"15min":29.731000000000002},"min":{"1min":11.714,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":281.464,"5min":281.464,"15min":695.39700000000005},"last":54.112000000000002},{"interface":"front","average":{"1min":23.632999999999999,"5min":18.774000000000001,"15min":26.581},"min":{"1min":9.2300000000000004,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":222.58799999999999,"5min":222.58799999999999,"15min":577.46299999999997},"last":22.428999999999998}]},{"osd":7,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":25.933,"5min":22.887,"15min":22.504999999999999},"min":{"1min":10.718999999999999,"5min":10.291,"15min":10.291},"max":{"1min":246.16200000000001,"5min":246.16200000000001,"15min":246.16200000000001},"last":50.445999999999998},{"interface":"front","average":{"1min":21.619,"5min":20.536000000000001,"15min":20.823},"min":{"1min":8.0589999999999993,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":173.136,"5min":173.136,"15min":173.136},"last":10.507999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542604,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64220,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307620,"statfs":{"total":96636764160,"available":96571002880,"internally_reserved":0,"allocated":11292672,"data_stored":6716024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":6.3659999999999997},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":8.0619999999999994}]},{"osd":1,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.869,"5min":15.02,"15min":851.14999999999998},"min":{"1min":7.2220000000000004,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":14.837999999999999,"5min":216.709,"15min":42105.105000000003},"last":18.609000000000002},{"interface":"front","average":{"1min":13.176,"5min":19.975999999999999,"15min":856.74199999999996},"min":{"1min":7.9480000000000004,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":17.890000000000001,"5min":217.929,"15min":42107.510000000002},"last":23.559000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":23.015999999999998},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":19.792999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.619,"5min":17.259,"15min":15.612},"min":{"1min":5.8739999999999997,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":16.402000000000001,"5min":149.886,"15min":172.952},"last":17.966000000000001},{"interface":"front","average":{"1min":15.183,"5min":24.073,"15min":18.233000000000001},"min":{"1min":10.442,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":25.030000000000001,"5min":231.691,"15min":231.691},"last":26.625}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19513.460999999999,"5min":8370.6499999999996,"15min":2802.931},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":8.5519999999999996},"max":{"1min":66230.460000000006,"5min":66734.365999999995,"15min":66734.365999999995},"last":20.399000000000001},{"interface":"front","average":{"1min":19500.227999999999,"5min":8373.2870000000003,"15min":2802.7069999999999},"min":{"1min":8.0199999999999996,"5min":7.282,"15min":7.1840000000000002},"max":{"1min":66215.267999999996,"5min":66719.994999999995,"15min":66719.994999999995},"last":27.327999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.661,"5min":21.093,"15min":2609.1860000000001},"min":{"1min":7.8140000000000001,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":24.298999999999999,"5min":247.161,"15min":74994.611999999994},"last":25.129000000000001},{"interface":"front","average":{"1min":13.272,"5min":20.702999999999999,"15min":2611.3249999999998},"min":{"1min":8.0670000000000002,"5min":8.0670000000000002,"15min":8.0670000000000002},"max":{"1min":21.920000000000002,"5min":241.91,"15min":74995.471000000005},"last":7.4080000000000004}]},{"osd":7,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":15.432,"5min":20.175999999999998,"15min":23.984999999999999},"min":{"1min":11.771000000000001,"5min":9.0150000000000006,"15min":9.0150000000000006},"max":{"1min":23.018000000000001,"5min":112.215,"15min":112.215},"last":19.236000000000001},{"interface":"front","average":{"1min":15.993,"5min":20.603000000000002,"15min":26.225999999999999},"min":{"1min":11.492000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":22.132999999999999,"5min":103.886,"15min":103.886},"last":22.161999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542605,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6361429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":9.8640000000000008,"5min":21.134,"15min":14.946},"min":{"1min":7.0199999999999996,"5min":4.8760000000000003,"15min":4.5640000000000001},"max":{"1min":16.805,"5min":302.62700000000001,"15min":302.62700000000001},"last":11.864000000000001},{"interface":"front","average":{"1min":13.249000000000001,"5min":24.818000000000001,"15min":18.748000000000001},"min":{"1min":8.6639999999999997,"5min":5.9379999999999997,"15min":5.9379999999999997},"max":{"1min":18.440999999999999,"5min":289.18200000000002,"15min":289.18200000000002},"last":17.309999999999999}]},{"osd":1,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":8.0830000000000002,"5min":26.745999999999999,"15min":1846.819},"min":{"1min":5.8609999999999998,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":12.907,"5min":682.15599999999995,"15min":57448.739999999998},"last":5.5780000000000003},{"interface":"front","average":{"1min":12.319000000000001,"5min":33.448,"15min":1853.625},"min":{"1min":8.2409999999999997,"5min":7.4470000000000001,"15min":7.4169999999999998},"max":{"1min":16.358000000000001,"5min":685.01300000000003,"15min":57479.919999999998},"last":9.2010000000000005}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":6.5359999999999996},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":11.598000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.497999999999999,"5min":28.012,"15min":18.911000000000001},"min":{"1min":8.6999999999999993,"5min":6.048,"15min":6.048},"max":{"1min":17.545000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":7.5499999999999998},{"interface":"front","average":{"1min":11.528,"5min":26.763000000000002,"15min":18.524000000000001},"min":{"1min":8.1560000000000006,"5min":6.4699999999999998,"15min":6.4050000000000002},"max":{"1min":18.917999999999999,"5min":609.31899999999996,"15min":609.31899999999996},"last":13.407}]},{"osd":5,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":14.292999999999999,"5min":40.664999999999999,"15min":26.305},"min":{"1min":8.8930000000000007,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":35.802,"5min":607.48199999999997,"15min":607.48199999999997},"last":12.811},{"interface":"front","average":{"1min":13.429,"5min":27.794,"15min":21.152999999999999},"min":{"1min":8.0269999999999992,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":37.572000000000003,"5min":287.95100000000002,"15min":592.32100000000003},"last":14.058}]},{"osd":6,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.945,"5min":26.32,"15min":2144.3380000000002},"min":{"1min":7.0549999999999997,"5min":7.0549999999999997,"15min":6.9480000000000004},"max":{"1min":16.335999999999999,"5min":688.24300000000005,"15min":75975.607000000004},"last":8.4900000000000002},{"interface":"front","average":{"1min":13.26,"5min":27.960999999999999,"15min":2143.8119999999999},"min":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.018000000000001,"5min":610.89800000000002,"15min":75971.296000000002},"last":14.66}]},{"osd":7,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":142.12799999999999,"15min":67.069000000000003},"min":{"1min":12.108000000000001,"5min":10.411,"15min":10.411},"max":{"1min":17.981000000000002,"5min":611.59799999999996,"15min":611.59799999999996},"last":15.207000000000001},{"interface":"front","average":{"1min":13.507999999999999,"5min":140.16900000000001,"15min":57.863},"min":{"1min":8.8960000000000008,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":18.225000000000001,"5min":629.07500000000005,"15min":629.07500000000005},"last":15.843999999999999}]}]},{"osd":3,"up_from":736,"seq":3161095930305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103948,"kb_used_data":10628,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267892,"statfs":{"total":96636764160,"available":96530321408,"internally_reserved":0,"allocated":10883072,"data_stored":6333298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.074999999999999,"5min":13.201000000000001,"15min":14.895},"min":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":101.90600000000001,"5min":136.09999999999999,"15min":159.828},"last":41.670000000000002},{"interface":"front","average":{"1min":12.532,"5min":13.233000000000001,"15min":14.949},"min":{"1min":6.0149999999999997,"5min":4.4000000000000004,"15min":4.4000000000000004},"max":{"1min":43.442999999999998,"5min":153.459,"15min":153.459},"last":40.825000000000003}]},{"osd":1,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":8.8539999999999992,"5min":13.349,"15min":386.524},"min":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":13.83,"5min":82.028000000000006,"15min":20852.026000000002},"last":152.60900000000001},{"interface":"front","average":{"1min":12.772,"5min":15.637,"15min":382.35199999999998},"min":{"1min":9.5039999999999996,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":27.445,"5min":141.65000000000001,"15min":20733.266},"last":61.618000000000002}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":192.08600000000001},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":179.96700000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":13.74,"5min":21.388999999999999,"15min":22.102},"min":{"1min":8.6859999999999999,"5min":7.7439999999999998,"15min":7.0129999999999999},"max":{"1min":31.504000000000001,"5min":194.845,"15min":201.905},"last":202.30099999999999},{"interface":"front","average":{"1min":14.554,"5min":21.510000000000002,"15min":22.814},"min":{"1min":8.1989999999999998,"5min":8.1199999999999992,"15min":6.944},"max":{"1min":33.781999999999996,"5min":196.65600000000001,"15min":210.239},"last":179.37299999999999}]},{"osd":5,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":18.071999999999999,"5min":21.088000000000001,"15min":24.696000000000002},"min":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":86.810000000000002,"5min":177.97999999999999,"15min":290.07100000000003},"last":180.82499999999999},{"interface":"front","average":{"1min":17.045000000000002,"5min":17.774999999999999,"15min":20.707000000000001},"min":{"1min":9.657,"5min":7.6619999999999999,"15min":6.8220000000000001},"max":{"1min":72.721999999999994,"5min":143.268,"15min":268.577},"last":117.535}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":17.827999999999999,"5min":18.077999999999999,"15min":1643.2470000000001},"min":{"1min":11.323,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":36.439999999999998,"5min":94.352000000000004,"15min":76086.231},"last":98.412999999999997},{"interface":"front","average":{"1min":15.978999999999999,"5min":20.367000000000001,"15min":1652.1669999999999},"min":{"1min":9.4280000000000008,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":56.807000000000002,"5min":146.048,"15min":76087.114000000001},"last":62.909999999999997}]},{"osd":7,"last update":"Thu Nov 2 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":16.010000000000002,"5min":19.359000000000002,"15min":23.548999999999999},"min":{"1min":10.382,"5min":8.8870000000000005,"15min":8.8870000000000005},"max":{"1min":40.081000000000003,"5min":152.32599999999999,"15min":152.32599999999999},"last":178.74299999999999},{"interface":"front","average":{"1min":16.681999999999999,"5min":19.952000000000002,"15min":24.228000000000002},"min":{"1min":9.5419999999999998,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":39.018000000000001,"5min":140.87299999999999,"15min":140.87299999999999},"last":118.634}]}]},{"osd":7,"up_from":875,"seq":3758096384428,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63908,"kb_used_data":10716,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307932,"statfs":{"total":96636764160,"available":96571322368,"internally_reserved":0,"allocated":10973184,"data_stored":6414677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":10.130000000000001,"5min":13.368,"15min":15.576000000000001},"min":{"1min":6.1070000000000002,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":15.327,"5min":84.426000000000002,"15min":84.426000000000002},"last":10.09},{"interface":"front","average":{"1min":8.7460000000000004,"5min":13.207000000000001,"15min":18.576000000000001},"min":{"1min":5.8650000000000002,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":14.715999999999999,"5min":97.984999999999999,"15min":97.984999999999999},"last":8.0839999999999996}]},{"osd":1,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":12.630000000000001,"5min":16.503,"15min":22.439},"min":{"1min":7.5309999999999997,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":16.297000000000001,"5min":178.56,"15min":178.56},"last":14.006},{"interface":"front","average":{"1min":9.2569999999999997,"5min":12.856999999999999,"15min":15.263},"min":{"1min":6.6589999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":11.67,"5min":66.081999999999994,"15min":66.081999999999994},"last":6.6159999999999997}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":7.5030000000000001},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":13.164999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":14.497999999999999,"5min":20.501000000000001,"15min":19.623000000000001},"min":{"1min":8.9700000000000006,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":25.792000000000002,"5min":253.71000000000001,"15min":253.71000000000001},"last":53.985999999999997},{"interface":"front","average":{"1min":12.202,"5min":16.843,"15min":18.457999999999998},"min":{"1min":7.9500000000000002,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":17.898,"5min":101.316,"15min":101.316},"last":73.027000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:44 2023","interfaces":[{"interface":"back","average":{"1min":15907.763999999999,"5min":9828.0190000000002,"15min":3302.183},"min":{"1min":8.968,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":65773.679000000004,"5min":66277.987999999998,"15min":66277.987999999998},"last":15.157999999999999},{"interface":"front","average":{"1min":15924.536,"5min":9833.3089999999993,"15min":3301.9609999999998},"min":{"1min":11.398999999999999,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":65797.989000000001,"5min":66301.380000000005,"15min":66301.380000000005},"last":16.475000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":13.279,"5min":21.350999999999999,"15min":22.777999999999999},"min":{"1min":8.9740000000000002,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":17.827999999999999,"5min":263.65699999999998,"15min":263.65699999999998},"last":11.058},{"interface":"front","average":{"1min":14.351000000000001,"5min":22.829999999999998,"15min":24.738},"min":{"1min":9.6669999999999998,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":19.327999999999999,"5min":183.96899999999999,"15min":183.96899999999999},"last":25.789999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":18.141999999999999,"5min":24.789000000000001,"15min":26.550999999999998},"min":{"1min":10.255000000000001,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":72.542000000000002,"5min":285.56999999999999,"15min":285.56999999999999},"last":12.564},{"interface":"front","average":{"1min":17.75,"5min":22.82,"15min":23.931999999999999},"min":{"1min":9.7289999999999992,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":28.934000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":15.885999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:13:23.521 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:13:23.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.620+0000 7f9af4451640 10 monclient: get_monmap_and_config 2023-11-02T16:13:23.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.620+0000 7f9af4451640 10 monclient: build_initial_monmap 2023-11-02T16:13:23.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.620+0000 7f9af4451640 10 monclient: monmap: 2023-11-02T16:13:23.623 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:23.623 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.623 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:23.621485+0000 2023-11-02T16:13:23.623 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:23.621485+0000 2023-11-02T16:13:23.624 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:23.624 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.624 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:23.624 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:23.624 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:23.625 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:23.625 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:23.625 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.621+0000 7f9af4451640 1 Processor -- start 2023-11-02T16:13:23.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.621+0000 7f9af4451640 1 -- start start 2023-11-02T16:13:23.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.621+0000 7f9af4451640 10 monclient: init 2023-11-02T16:13:23.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:23.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2023-11-02T16:13:23.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9aec0b4260 0x7f9aec0a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-a con 0x7f9aec0b4260 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:23.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9aec141730 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:23.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9aec13e520 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:23.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec0a9210 con 0x7f9aec13e520 2023-11-02T16:13:23.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec1422b0 con 0x7f9aec141730 2023-11-02T16:13:23.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0aa550 con 0x7f9aec0b4260 2023-11-02T16:13:23.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:23.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af4451640 10 monclient(hunting): authenticate will time out at 3730.963816s 2023-11-02T16:13:23.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.622+0000 7f9af21ed640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9aec0b4260 0x7f9aec0a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af19ec640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9aec141730 0x7f9aec141ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:38464/0 (socket says 172.21.15.174:38464) 2023-11-02T16:13:23.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af19ec640 1 -- 172.21.15.174:0/1666158138 learned_addr learned my addr 172.21.15.174:0/1666158138 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:23.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): get_auth_request con 0x7f9aec0b4260 auth_method 0 2023-11-02T16:13:23.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.623+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 10 monclient(hunting): handle_auth_done global_id 15774 payload 274 2023-11-02T16:13:23.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 1 -- 172.21.15.174:0/1666158138 >> v1:172.21.15.175:6789/0 conn(0x7f9aec141730 legacy=0x7f9aec141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 1 -- 172.21.15.174:0/1666158138 >> v1:172.21.15.174:6789/0 conn(0x7f9aec13e520 legacy=0x7f9aec13e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:23.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 10 monclient: _finish_hunting 0 2023-11-02T16:13:23.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 1 monclient: found mon.noname-a 2023-11-02T16:13:23.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:23.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:23.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.625+0000 7f9af21ed640 1 -- 172.21.15.174:0/1666158138 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0aa750 con 0x7f9aec0b4260 2023-11-02T16:13:23.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.627+0000 7f9af21ed640 1 --2- 172.21.15.174:0/1666158138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec0b4260 0x7f9aec0a8870 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80097d0 tx=0x7f9ae802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18826c3dd33f1e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 1 -- 172.21.15.174:0/1666158138 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae802e070 con 0x7f9aec0b4260 2023-11-02T16:13:23.639 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.640 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:23.640 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: dump: 2023-11-02T16:13:23.640 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.641 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.641 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.641 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.641 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.642 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.642 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.642 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.642 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.643 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.643 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: _finish_auth 0 2023-11-02T16:13:23.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: _check_auth_tickets 2023-11-02T16:13:23.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:23.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 1 -- 172.21.15.174:0/1666158138 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80315a0 con 0x7f9aec0b4260 2023-11-02T16:13:23.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af4451640 5 monclient: authenticate success, global_id 15774 2023-11-02T16:13:23.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af4451640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:23.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:23.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af4451640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:23.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 1 -- 172.21.15.174:0/1666158138 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8031930 con 0x7f9aec0b4260 2023-11-02T16:13:23.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af4451640 4 set_mon_vals no callback set 2023-11-02T16:13:23.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.631+0000 7f9af11eb640 10 monclient: dump: 2023-11-02T16:13:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.648 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.648 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.648 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.648 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.648 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.649 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 10 monclient: shutdown 2023-11-02T16:13:23.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:23.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 -- 172.21.15.174:0/1666158138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec0b4260 msgr2=0x7f9aec0a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 --2- 172.21.15.174:0/1666158138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec0b4260 0x7f9aec0a8870 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80097d0 tx=0x7f9ae802b840 comp rx=0 tx=0).stop 2023-11-02T16:13:23.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 -- 172.21.15.174:0/1666158138 shutdown_connections 2023-11-02T16:13:23.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 --2- 172.21.15.174:0/1666158138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec0b4260 0x7f9aec0a8870 unknown :-1 s=CLOSED pgs=3187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 -- 172.21.15.174:0/1666158138 >> 172.21.15.174:0/1666158138 conn(0x7f9aec065d50 msgr2=0x7f9aec066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:23.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 -- 172.21.15.174:0/1666158138 shutdown_connections 2023-11-02T16:13:23.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.632+0000 7f9af4451640 1 -- 172.21.15.174:0/1666158138 wait complete. 2023-11-02T16:13:23.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 10 monclient: build_initial_monmap 2023-11-02T16:13:23.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 10 monclient: monmap: 2023-11-02T16:13:23.651 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:23.651 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:23.651 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:23.657 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:23.657 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:23.658 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.658 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:23.658 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:23.658 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:23.659 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 librados: starting msgr at 2023-11-02T16:13:23.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 librados: starting objecter 2023-11-02T16:13:23.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 Processor -- start 2023-11-02T16:13:23.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 -- start start 2023-11-02T16:13:23.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 librados: setting wanted keys 2023-11-02T16:13:23.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 1 librados: calling monclient init 2023-11-02T16:13:23.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.633+0000 7f9af4451640 10 monclient: init 2023-11-02T16:13:23.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:23.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T16:13:23.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aec0b4260 0x7f9aec139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-b con 0x7f9aec0b4260 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:23.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-a con 0x7f9aec13e520 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:23.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9aec141730 0x7f9aec13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): picked mon.noname-c con 0x7f9aec141730 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:23.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec0ab990 con 0x7f9aec13e520 2023-11-02T16:13:23.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec140970 con 0x7f9aec0b4260 2023-11-02T16:13:23.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:23.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1cb850 con 0x7f9aec141730 2023-11-02T16:13:23.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:23.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af4451640 10 monclient(hunting): authenticate will time out at 3730.976059s 2023-11-02T16:13:23.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.634+0000 7f9af19ec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9aec141730 0x7f9aec13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.664 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44520/0 (socket says 172.21.15.174:44520) 2023-11-02T16:13:23.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 1 -- 172.21.15.174:0/2438066992 learned_addr learned my addr 172.21.15.174:0/2438066992 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:23.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): get_auth_request con 0x7f9aec13e520 auth_method 0 2023-11-02T16:13:23.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): get_auth_request con 0x7f9aec141730 auth_method 0 2023-11-02T16:13:23.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af21ed640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aec0b4260 0x7f9aec139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af21ed640 10 monclient(hunting): get_auth_request con 0x7f9aec0b4260 auth_method 0 2023-11-02T16:13:23.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af21ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:23.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af21ed640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:23.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af21ed640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:23.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af19ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.635+0000 7f9af29ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.636+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:23.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.636+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:23.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.636+0000 7f9af21ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:23.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 10 monclient(hunting): handle_auth_done global_id 15777 payload 824 2023-11-02T16:13:23.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 -- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9aec141730 msgr2=0x7f9aec13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9aec141730 0x7f9aec13fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 -- 172.21.15.174:0/2438066992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aec0b4260 msgr2=0x7f9aec139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:23.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aec0b4260 0x7f9aec139080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:23.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 10 monclient: _finish_hunting 0 2023-11-02T16:13:23.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 monclient: found mon.noname-a 2023-11-02T16:13:23.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:23.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:23.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.638+0000 7f9af19ec640 1 -- 172.21.15.174:0/2438066992 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009400 con 0x7f9aec13e520 2023-11-02T16:13:23.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.640+0000 7f9af19ec640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c480 tx=0x7f9adc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.640+0000 7f9ae2ffd640 10 client.?.objecter ms_handle_connect 0x7f9aec13e520 2023-11-02T16:13:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.640+0000 7f9ae2ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc005ce0 con 0x7f9aec13e520 2023-11-02T16:13:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: dump: 2023-11-02T16:13:23.679 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.679 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.679 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.679 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.679 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.680 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.680 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.680 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.680 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.680 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:23.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:23.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:23.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc005e40 con 0x7f9aec13e520 2023-11-02T16:13:23.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:23.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0072f0 con 0x7f9aec13e520 2023-11-02T16:13:23.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 5 monclient: authenticate success, global_id 15777 2023-11-02T16:13:23.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 10 monclient: _renew_subs 2023-11-02T16:13:23.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:23.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec1411e0 con 0x7f9aec13e520 2023-11-02T16:13:23.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:23.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af09ea640 4 set_mon_vals no callback set 2023-11-02T16:13:23.683 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:23.684 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9ae2ffd640 10 monclient: dump: 2023-11-02T16:13:23.684 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:23.684 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:23.685 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.685 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:23.685 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:23.685 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:23.685 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:23.686 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:23.686 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:23.686 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:23.686 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 10 client.15777.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:23.686 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 10 monclient: _renew_subs 2023-11-02T16:13:23.687 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:23.687 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec140450 con 0x7f9aec13e520 2023-11-02T16:13:23.687 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.644+0000 7f9af4451640 1 librados: init done 2023-11-02T16:13:23.687 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.645+0000 7f9af4451640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.645+0000 7f9af4451640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.645+0000 7f9af4451640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:23.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.645+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec140b30 con 0x7f9aec13e520 2023-11-02T16:13:23.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.892+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9adc007510 con 0x7f9aec13e520 2023-11-02T16:13:23.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.892+0000 7f9ae2ffd640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 0x7f9ad006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:23.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.892+0000 7f9af21ed640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 0x7f9ad006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:23.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.892+0000 7f9af21ed640 10 monclient: get_auth_request con 0x7f9ad006be10 auth_method 0 2023-11-02T16:13:23.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.893+0000 7f9ae2ffd640 10 client.15777.objecter ms_handle_connect 0x7f9ad006be10 2023-11-02T16:13:23.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.893+0000 7f9af21ed640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 0x7f9ad006e2d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8036000 tx=0x7f9ae803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:23.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1029..1029 src has 247..1029) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f9adc057680 con 0x7f9aec13e520 2023-11-02T16:13:23.908 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 10 client.15777.objecter ms_dispatch 0x7f9aec000b90 osd_map(1029..1029 src has 247..1029) v4 2023-11-02T16:13:23.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 3 client.15777.objecter handle_osd_map got epochs [1029,1029] > 0 2023-11-02T16:13:23.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 3 client.15777.objecter handle_osd_map decoding full epoch 1029 2023-11-02T16:13:23.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 20 client.15777.objecter dump_active .. 0 homeless 2023-11-02T16:13:23.909 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9adc0539c0 con 0x7f9aec13e520 2023-11-02T16:13:23.910 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:23.910 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:23.906+0000 7f9ae2ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:24.078 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.073+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9aec0a9680 con 0x7f9ad006be10 2023-11-02T16:13:24.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.077+0000 7f9ae2ffd640 1 -- 172.21.15.174:0/2438066992 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152018 (secure 0 0 0) 0x7f9aec0a9680 con 0x7f9ad006be10 2023-11-02T16:13:24.081 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:24.082 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:13:24.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 10 librados: watch_flush enter 2023-11-02T16:13:24.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 10 librados: watch_flush exit 2023-11-02T16:13:24.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 20 client.15777.objecter shutdown clearing up homeless session... 2023-11-02T16:13:24.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 10 client.15777.objecter successfully canceled tick 2023-11-02T16:13:24.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 msgr2=0x7f9ad006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 0x7f9ad006e2d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8036000 tx=0x7f9ae803d040 comp rx=0 tx=0).stop 2023-11-02T16:13:24.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 10 monclient: shutdown 2023-11-02T16:13:24.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:24.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 msgr2=0x7f9aec1cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c480 tx=0x7f9adc00c940 comp rx=0 tx=0).stop 2023-11-02T16:13:24.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 shutdown_connections 2023-11-02T16:13:24.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f9ad006be10 0x7f9ad006e2d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9aec141730 0x7f9aec13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9aec13e520 0x7f9aec1cb0b0 unknown :-1 s=CLOSED pgs=3188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 --2- 172.21.15.174:0/2438066992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aec0b4260 0x7f9aec139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 >> 172.21.15.174:0/2438066992 conn(0x7f9aec065d50 msgr2=0x7f9aec13b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:24.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 shutdown_connections 2023-11-02T16:13:24.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 -- 172.21.15.174:0/2438066992 wait complete. 2023-11-02T16:13:24.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.080+0000 7f9af4451640 1 librados: shutdown 2023-11-02T16:13:24.094 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2218,"stamp":"2023-11-02T16:13:23.403888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609908,"kb_used_data":86068,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364812,"statfs":{"total":773094113280,"available":772469567488,"internally_reserved":0,"allocated":88133632,"data_stored":51603970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":432,"apply_latency_ms":432,"commit_latency_ns":432000000,"apply_latency_ns":432000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.400285"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":998,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.956626+0000","last_change":"2023-11-02T16:13:17.157671+0000","last_active":"2023-11-02T16:13:17.956626+0000","last_peered":"2023-11-02T16:13:17.956626+0000","last_clean":"2023-11-02T16:13:17.956626+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:17.956626+0000","last_undegraded":"2023-11-02T16:13:17.956626+0000","last_fullsized":"2023-11-02T16:13:17.956626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:58:17.271177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.749561+0000","last_change":"2023-11-02T16:07:27.307241+0000","last_active":"2023-11-02T16:13:17.749561+0000","last_peered":"2023-11-02T16:13:17.749561+0000","last_clean":"2023-11-02T16:13:17.749561+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:17.749561+0000","last_undegraded":"2023-11-02T16:13:17.749561+0000","last_fullsized":"2023-11-02T16:13:17.749561+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:12:49.486828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.186771+0000","last_change":"2023-11-02T16:07:27.395825+0000","last_active":"2023-11-02T16:13:21.186771+0000","last_peered":"2023-11-02T16:13:21.186771+0000","last_clean":"2023-11-02T16:13:21.186771+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:21.186771+0000","last_undegraded":"2023-11-02T16:13:21.186771+0000","last_fullsized":"2023-11-02T16:13:21.186771+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:25:43.494367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":972,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.121930+0000","last_change":"2023-11-02T15:35:17.345372+0000","last_active":"2023-11-02T16:13:19.121930+0000","last_peered":"2023-11-02T16:13:19.121930+0000","last_clean":"2023-11-02T16:13:19.121930+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:19.121930+0000","last_undegraded":"2023-11-02T16:13:19.121930+0000","last_fullsized":"2023-11-02T16:13:19.121930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:39:06.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.159925+0000","last_change":"2023-11-02T15:35:14.211126+0000","last_active":"2023-11-02T16:13:21.159925+0000","last_peered":"2023-11-02T16:13:21.159925+0000","last_clean":"2023-11-02T16:13:21.159925+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:21.159925+0000","last_undegraded":"2023-11-02T16:13:21.159925+0000","last_fullsized":"2023-11-02T16:13:21.159925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:19:16.985396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":920,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.171904+0000","last_change":"2023-11-02T15:59:17.698985+0000","last_active":"2023-11-02T16:13:21.171904+0000","last_peered":"2023-11-02T16:13:21.171904+0000","last_clean":"2023-11-02T16:13:21.171904+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:21.171904+0000","last_undegraded":"2023-11-02T16:13:21.171904+0000","last_fullsized":"2023-11-02T16:13:21.171904+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:31:01.071757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":998,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.146603+0000","last_change":"2023-11-02T16:13:16.820908+0000","last_active":"2023-11-02T16:13:21.146603+0000","last_peered":"2023-11-02T16:13:21.146603+0000","last_clean":"2023-11-02T16:13:21.146603+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:21.146603+0000","last_undegraded":"2023-11-02T16:13:21.146603+0000","last_fullsized":"2023-11-02T16:13:21.146603+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:08:51.272032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.143826+0000","last_change":"2023-11-02T15:35:30.105402+0000","last_active":"2023-11-02T16:13:21.143826+0000","last_peered":"2023-11-02T16:13:21.143826+0000","last_clean":"2023-11-02T16:13:21.143826+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:21.143826+0000","last_undegraded":"2023-11-02T16:13:21.143826+0000","last_fullsized":"2023-11-02T16:13:21.143826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:00:16.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":960,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.061767+0000","last_change":"2023-11-02T16:07:27.407922+0000","last_active":"2023-11-02T16:13:20.061767+0000","last_peered":"2023-11-02T16:13:20.061767+0000","last_clean":"2023-11-02T16:13:20.061767+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:20.061767+0000","last_undegraded":"2023-11-02T16:13:20.061767+0000","last_fullsized":"2023-11-02T16:13:20.061767+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:16:50.921541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1001,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.087417+0000","last_change":"2023-11-02T16:13:16.885325+0000","last_active":"2023-11-02T16:13:20.087417+0000","last_peered":"2023-11-02T16:13:20.087417+0000","last_clean":"2023-11-02T16:13:20.087417+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:20.087417+0000","last_undegraded":"2023-11-02T16:13:20.087417+0000","last_fullsized":"2023-11-02T16:13:20.087417+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:18:37.960941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1011,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.069466+0000","last_change":"2023-11-02T16:13:16.913467+0000","last_active":"2023-11-02T16:13:20.069466+0000","last_peered":"2023-11-02T16:13:20.069466+0000","last_clean":"2023-11-02T16:13:20.069466+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:20.069466+0000","last_undegraded":"2023-11-02T16:13:20.069466+0000","last_fullsized":"2023-11-02T16:13:20.069466+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:05:41.639058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.719637+0000","last_change":"2023-11-02T16:13:17.189587+0000","last_active":"2023-11-02T16:13:17.719637+0000","last_peered":"2023-11-02T16:13:17.719637+0000","last_clean":"2023-11-02T16:13:17.719637+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:17.719637+0000","last_undegraded":"2023-11-02T16:13:17.719637+0000","last_fullsized":"2023-11-02T16:13:17.719637+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:04:14.846503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":983,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.746564+0000","last_change":"2023-11-02T15:59:17.640186+0000","last_active":"2023-11-02T16:13:17.746564+0000","last_peered":"2023-11-02T16:13:17.746564+0000","last_clean":"2023-11-02T16:13:17.746564+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:17.746564+0000","last_undegraded":"2023-11-02T16:13:17.746564+0000","last_fullsized":"2023-11-02T16:13:17.746564+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:57.212864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.744965+0000","last_change":"2023-11-02T15:59:17.577430+0000","last_active":"2023-11-02T16:13:17.744965+0000","last_peered":"2023-11-02T16:13:17.744965+0000","last_clean":"2023-11-02T16:13:17.744965+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:17.744965+0000","last_undegraded":"2023-11-02T16:13:17.744965+0000","last_fullsized":"2023-11-02T16:13:17.744965+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:03:26.273290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":973,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.959279+0000","last_change":"2023-11-02T15:35:23.837202+0000","last_active":"2023-11-02T16:13:17.959279+0000","last_peered":"2023-11-02T16:13:17.959279+0000","last_clean":"2023-11-02T16:13:17.959279+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:17.959279+0000","last_undegraded":"2023-11-02T16:13:17.959279+0000","last_fullsized":"2023-11-02T16:13:17.959279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:24.083855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":978,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.094447+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T16:13:19.094447+0000","last_peered":"2023-11-02T16:13:19.094447+0000","last_clean":"2023-11-02T16:13:19.094447+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:19.094447+0000","last_undegraded":"2023-11-02T16:13:19.094447+0000","last_fullsized":"2023-11-02T16:13:19.094447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.954242+0000","last_change":"2023-11-02T15:59:17.883690+0000","last_active":"2023-11-02T16:13:17.954242+0000","last_peered":"2023-11-02T16:13:17.954242+0000","last_clean":"2023-11-02T16:13:17.954242+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:17.954242+0000","last_undegraded":"2023-11-02T16:13:17.954242+0000","last_fullsized":"2023-11-02T16:13:17.954242+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:58:33.336681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":989,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.718156+0000","last_change":"2023-11-02T15:59:17.612075+0000","last_active":"2023-11-02T16:13:17.718156+0000","last_peered":"2023-11-02T16:13:17.718156+0000","last_clean":"2023-11-02T16:13:17.718156+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:17.718156+0000","last_undegraded":"2023-11-02T16:13:17.718156+0000","last_fullsized":"2023-11-02T16:13:17.718156+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:08:04.600098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1013,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.071639+0000","last_change":"2023-11-02T16:13:17.151154+0000","last_active":"2023-11-02T16:13:20.071639+0000","last_peered":"2023-11-02T16:13:20.071639+0000","last_clean":"2023-11-02T16:13:20.071639+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:20.071639+0000","last_undegraded":"2023-11-02T16:13:20.071639+0000","last_fullsized":"2023-11-02T16:13:20.071639+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:40.667953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":952,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.080989+0000","last_change":"2023-11-02T15:43:56.978148+0000","last_active":"2023-11-02T16:13:20.080989+0000","last_peered":"2023-11-02T16:13:20.080989+0000","last_clean":"2023-11-02T16:13:20.080989+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:20.080989+0000","last_undegraded":"2023-11-02T16:13:20.080989+0000","last_fullsized":"2023-11-02T16:13:20.080989+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:17:20.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":530,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.732985+0000","last_change":"2023-11-02T16:07:27.487367+0000","last_active":"2023-11-02T16:13:17.732985+0000","last_peered":"2023-11-02T16:13:17.732985+0000","last_clean":"2023-11-02T16:13:17.732985+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:17.732985+0000","last_undegraded":"2023-11-02T16:13:17.732985+0000","last_fullsized":"2023-11-02T16:13:17.732985+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:15:14.310785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1004,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.696771+0000","last_change":"2023-11-02T15:59:17.535951+0000","last_active":"2023-11-02T16:13:17.696771+0000","last_peered":"2023-11-02T16:13:17.696771+0000","last_clean":"2023-11-02T16:13:17.696771+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:17.696771+0000","last_undegraded":"2023-11-02T16:13:17.696771+0000","last_fullsized":"2023-11-02T16:13:17.696771+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:50:51.454223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":975,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.934494+0000","last_change":"2023-11-02T15:35:16.434956+0000","last_active":"2023-11-02T16:13:17.934494+0000","last_peered":"2023-11-02T16:13:17.934494+0000","last_clean":"2023-11-02T16:13:17.934494+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:17.934494+0000","last_undegraded":"2023-11-02T16:13:17.934494+0000","last_fullsized":"2023-11-02T16:13:17.934494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:54:41.093040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":986,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.729224+0000","last_change":"2023-11-02T16:07:27.307462+0000","last_active":"2023-11-02T16:13:17.729224+0000","last_peered":"2023-11-02T16:13:17.729224+0000","last_clean":"2023-11-02T16:13:17.729224+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:17.729224+0000","last_undegraded":"2023-11-02T16:13:17.729224+0000","last_fullsized":"2023-11-02T16:13:17.729224+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:53:09.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":935,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.988390+0000","last_change":"2023-11-02T15:35:10.705051+0000","last_active":"2023-11-02T16:13:19.988390+0000","last_peered":"2023-11-02T16:13:19.988390+0000","last_clean":"2023-11-02T16:13:19.988390+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:19.988390+0000","last_undegraded":"2023-11-02T16:13:19.988390+0000","last_fullsized":"2023-11-02T16:13:19.988390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:36:19.002698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":990,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.703806+0000","last_change":"2023-11-02T15:59:17.407544+0000","last_active":"2023-11-02T16:13:17.703806+0000","last_peered":"2023-11-02T16:13:17.703806+0000","last_clean":"2023-11-02T16:13:17.703806+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:17.703806+0000","last_undegraded":"2023-11-02T16:13:17.703806+0000","last_fullsized":"2023-11-02T16:13:17.703806+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:52:17.493086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":973,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.156486+0000","last_change":"2023-11-02T15:35:30.088014+0000","last_active":"2023-11-02T16:13:21.156486+0000","last_peered":"2023-11-02T16:13:21.156486+0000","last_clean":"2023-11-02T16:13:21.156486+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:21.156486+0000","last_undegraded":"2023-11-02T16:13:21.156486+0000","last_fullsized":"2023-11-02T16:13:21.156486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:24:59.243202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.710169+0000","last_change":"2023-11-02T16:13:17.216261+0000","last_active":"2023-11-02T16:13:17.710169+0000","last_peered":"2023-11-02T16:13:17.710169+0000","last_clean":"2023-11-02T16:13:17.710169+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:17.710169+0000","last_undegraded":"2023-11-02T16:13:17.710169+0000","last_fullsized":"2023-11-02T16:13:17.710169+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:51:41.117006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"22'76","reported_seq":1085,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.149378+0000","last_change":"2023-11-02T15:33:55.693910+0000","last_active":"2023-11-02T16:13:21.149378+0000","last_peered":"2023-11-02T16:13:21.149378+0000","last_clean":"2023-11-02T16:13:21.149378+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:21.149378+0000","last_undegraded":"2023-11-02T16:13:21.149378+0000","last_fullsized":"2023-11-02T16:13:21.149378+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_clean_scrub_stamp":"2023-11-02T15:33:53.161912+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:29.329385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"40'7","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.936228+0000","last_change":"2023-11-02T16:13:17.029849+0000","last_active":"2023-11-02T16:13:17.936228+0000","last_peered":"2023-11-02T16:13:17.936228+0000","last_clean":"2023-11-02T16:13:17.936228+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:17.936228+0000","last_undegraded":"2023-11-02T16:13:17.936228+0000","last_fullsized":"2023-11-02T16:13:17.936228+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:45:56.714871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1002,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.038464+0000","last_change":"2023-11-02T16:13:17.007381+0000","last_active":"2023-11-02T16:13:20.038464+0000","last_peered":"2023-11-02T16:13:20.038464+0000","last_clean":"2023-11-02T16:13:20.038464+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:20.038464+0000","last_undegraded":"2023-11-02T16:13:20.038464+0000","last_fullsized":"2023-11-02T16:13:20.038464+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:29:31.495127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1002,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.079106+0000","last_change":"2023-11-02T16:13:16.951026+0000","last_active":"2023-11-02T16:13:19.079106+0000","last_peered":"2023-11-02T16:13:19.079106+0000","last_clean":"2023-11-02T16:13:19.079106+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:19.079106+0000","last_undegraded":"2023-11-02T16:13:19.079106+0000","last_fullsized":"2023-11-02T16:13:19.079106+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:01:56.841900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":962,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.028826+0000","last_change":"2023-11-02T16:13:17.159877+0000","last_active":"2023-11-02T16:13:20.028826+0000","last_peered":"2023-11-02T16:13:20.028826+0000","last_clean":"2023-11-02T16:13:20.028826+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:20.028826+0000","last_undegraded":"2023-11-02T16:13:20.028826+0000","last_fullsized":"2023-11-02T16:13:20.028826+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:32:03.253445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":987,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.120389+0000","last_change":"2023-11-02T15:43:57.129812+0000","last_active":"2023-11-02T16:13:19.120389+0000","last_peered":"2023-11-02T16:13:19.120389+0000","last_clean":"2023-11-02T16:13:19.120389+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:19.120389+0000","last_undegraded":"2023-11-02T16:13:19.120389+0000","last_fullsized":"2023-11-02T16:13:19.120389+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:42.294660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":976,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.092801+0000","last_change":"2023-11-02T15:35:17.386786+0000","last_active":"2023-11-02T16:13:19.092801+0000","last_peered":"2023-11-02T16:13:19.092801+0000","last_clean":"2023-11-02T16:13:19.092801+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:19.092801+0000","last_undegraded":"2023-11-02T16:13:19.092801+0000","last_fullsized":"2023-11-02T16:13:19.092801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:58:39.849736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":984,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.748036+0000","last_change":"2023-11-02T15:59:17.643298+0000","last_active":"2023-11-02T16:13:17.748036+0000","last_peered":"2023-11-02T16:13:17.748036+0000","last_clean":"2023-11-02T16:13:17.748036+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:17.748036+0000","last_undegraded":"2023-11-02T16:13:17.748036+0000","last_fullsized":"2023-11-02T16:13:17.748036+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:31:03.424299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":985,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.725844+0000","last_change":"2023-11-02T15:59:17.471575+0000","last_active":"2023-11-02T16:13:17.725844+0000","last_peered":"2023-11-02T16:13:17.725844+0000","last_clean":"2023-11-02T16:13:17.725844+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:17.725844+0000","last_undegraded":"2023-11-02T16:13:17.725844+0000","last_fullsized":"2023-11-02T16:13:17.725844+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:21.803164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.183490+0000","last_change":"2023-11-02T16:13:16.821060+0000","last_active":"2023-11-02T16:13:21.183490+0000","last_peered":"2023-11-02T16:13:21.183490+0000","last_clean":"2023-11-02T16:13:21.183490+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:21.183490+0000","last_undegraded":"2023-11-02T16:13:21.183490+0000","last_fullsized":"2023-11-02T16:13:21.183490+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:46.475704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.055317+0000","last_change":"2023-11-02T16:07:27.399974+0000","last_active":"2023-11-02T16:13:19.055317+0000","last_peered":"2023-11-02T16:13:19.055317+0000","last_clean":"2023-11-02T16:13:19.055317+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:19.055317+0000","last_undegraded":"2023-11-02T16:13:19.055317+0000","last_fullsized":"2023-11-02T16:13:19.055317+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:02:28.668857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.077626+0000","last_change":"2023-11-02T16:13:16.955603+0000","last_active":"2023-11-02T16:13:19.077626+0000","last_peered":"2023-11-02T16:13:19.077626+0000","last_clean":"2023-11-02T16:13:19.077626+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:19.077626+0000","last_undegraded":"2023-11-02T16:13:19.077626+0000","last_fullsized":"2023-11-02T16:13:19.077626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:12:56.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":997,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.158342+0000","last_change":"2023-11-02T16:13:16.834330+0000","last_active":"2023-11-02T16:13:21.158342+0000","last_peered":"2023-11-02T16:13:21.158342+0000","last_clean":"2023-11-02T16:13:21.158342+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:21.158342+0000","last_undegraded":"2023-11-02T16:13:21.158342+0000","last_fullsized":"2023-11-02T16:13:21.158342+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:56:31.478240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544251,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63832,"kb_used_data":10640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308008,"statfs":{"total":96636764160,"available":96571400192,"internally_reserved":0,"allocated":10895360,"data_stored":6318201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":15.131},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":6.7720000000000002}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":9.5830000000000002},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":13.759}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":10.568},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":8.0370000000000008}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":16.782},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":18.870000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":11.778},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":14.539999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":15.752000000000001},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":11.177}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":18.274999999999999},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":12.448}]}]},{"osd":1,"up_from":16,"seq":68719477187,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104348,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267492,"statfs":{"total":96636764160,"available":96529911808,"internally_reserved":0,"allocated":11292672,"data_stored":6716036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":8.7509999999999994,"5min":11.885,"15min":17.582000000000001},"min":{"1min":4.5650000000000004,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":13.523999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":13.021000000000001},{"interface":"front","average":{"1min":8.1389999999999993,"5min":12.238,"15min":18.489999999999998},"min":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":11.058999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":13.617000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":28.122},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":27.352}]},{"osd":3,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":11.907,"5min":13.795999999999999,"15min":21.059999999999999},"min":{"1min":8.5370000000000008,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":26.161000000000001,"5min":77.778000000000006,"15min":549.92200000000003},"last":11.098000000000001},{"interface":"front","average":{"1min":12.789999999999999,"5min":17.585999999999999,"15min":24.571000000000002},"min":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":7.8280000000000003},"max":{"1min":26.827000000000002,"5min":108.244,"15min":542.60500000000002},"last":7.3369999999999997}]},{"osd":4,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":107.46299999999999,"5min":110.161,"15min":49.548000000000002},"min":{"1min":7.8819999999999997,"5min":7.8819999999999997,"15min":6.1360000000000001},"max":{"1min":1884.9880000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":9.391},{"interface":"front","average":{"1min":111.589,"5min":110.36199999999999,"15min":51.369},"min":{"1min":7.6950000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"max":{"1min":1970.0229999999999,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":8.8599999999999994}]},{"osd":5,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":16.436,"15min":25.068999999999999},"min":{"1min":8.0990000000000002,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":16.541,"5min":71.225999999999999,"15min":529.93499999999995},"last":36.536999999999999},{"interface":"front","average":{"1min":15.509,"5min":19.974,"15min":26.859000000000002},"min":{"1min":9.5839999999999996,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":23.559999999999999,"5min":122.14,"15min":555.77700000000004},"last":29.494}]},{"osd":6,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":15.324999999999999,"5min":22.655000000000001,"15min":1553.2819999999999},"min":{"1min":9.0719999999999992,"5min":8.6210000000000004,"15min":8.1400000000000006},"max":{"1min":31.744,"5min":245.36500000000001,"15min":73455.679000000004},"last":14.265000000000001},{"interface":"front","average":{"1min":15.226000000000001,"5min":17.460999999999999,"15min":1549.6890000000001},"min":{"1min":9.5269999999999992,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":30.879999999999999,"5min":124.36499999999999,"15min":73453.190000000002},"last":11.747999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:12:35 2023","interfaces":[{"interface":"back","average":{"1min":15.096,"5min":18.516999999999999,"15min":21.047000000000001},"min":{"1min":10.757999999999999,"5min":10.097,"15min":10.097},"max":{"1min":21.617999999999999,"5min":91.019999999999996,"15min":91.019999999999996},"last":9.9969999999999999},{"interface":"front","average":{"1min":16.209,"5min":22.247,"15min":24.065999999999999},"min":{"1min":10.824999999999999,"5min":10.113,"15min":10.113},"max":{"1min":28.861999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":15.321999999999999}]}]},{"osd":0,"up_from":16,"seq":68719477211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81968,"kb_used_data":10728,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289872,"statfs":{"total":96636764160,"available":96552828928,"internally_reserved":0,"allocated":10985472,"data_stored":6426109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:12:55 2023","interfaces":[{"interface":"back","average":{"1min":8.7309999999999999,"5min":13.488,"15min":77.375},"min":{"1min":5.3490000000000002,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.488,"5min":107.59999999999999,"15min":5781.2129999999997},"last":7.5629999999999997},{"interface":"front","average":{"1min":8.4719999999999995,"5min":11.718,"15min":74.570999999999998},"min":{"1min":6.2249999999999996,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":13.672000000000001,"5min":73.094999999999999,"15min":5443.4889999999996},"last":11.455}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.778},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":13.651999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":12.1,"5min":17.452999999999999,"15min":17.625},"min":{"1min":8.3399999999999999,"5min":7.46,"15min":7.46},"max":{"1min":29.192,"5min":156.99100000000001,"15min":234.29300000000001},"last":40.524999999999999},{"interface":"front","average":{"1min":12.449999999999999,"5min":17.09,"15min":15.265000000000001},"min":{"1min":7.6239999999999997,"5min":7.4950000000000001,"15min":6.6070000000000002},"max":{"1min":34.267000000000003,"5min":151.30099999999999,"15min":151.30099999999999},"last":62.927}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":21213.947,"5min":17485.800999999999,"15min":5840.5320000000002},"min":{"1min":9.7330000000000005,"5min":7.1600000000000001,"15min":6.3200000000000003},"max":{"1min":64456.964,"5min":66161.077000000005,"15min":66161.077000000005},"last":14.664},{"interface":"front","average":{"1min":21222.184000000001,"5min":17490.220000000001,"15min":5843.1549999999997},"min":{"1min":11.762,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":64460.497000000003,"5min":66163.926000000007,"15min":66163.926000000007},"last":15.914}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":16.919},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":17.655000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":18.556999999999999,"5min":22.667000000000002,"15min":1772.4739999999999},"min":{"1min":14.353999999999999,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":38.277000000000001,"5min":107.51000000000001,"15min":71055.725999999995},"last":18.425000000000001},{"interface":"front","average":{"1min":13.893000000000001,"5min":19.393000000000001,"15min":1770.963},"min":{"1min":7.7320000000000002,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":32.561,"5min":156.91,"15min":71055.100000000006},"last":15.398}]},{"osd":7,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19.140000000000001,"5min":25.041,"15min":30.241},"min":{"1min":13.130000000000001,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":44.636000000000003,"5min":205.88999999999999,"15min":205.88999999999999},"last":19.449000000000002},{"interface":"front","average":{"1min":11.23,"5min":17.503,"15min":22.859000000000002},"min":{"1min":8.0190000000000001,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":31.449999999999999,"5min":155.96600000000001,"15min":155.96600000000001},"last":12.237}]}]},{"osd":6,"up_from":387,"seq":1662152343954,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63828,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308012,"statfs":{"total":96636764160,"available":96571404288,"internally_reserved":0,"allocated":10891264,"data_stored":6318196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.405000000000001,"5min":14.585000000000001,"15min":20.376999999999999},"min":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.218},"max":{"1min":208.60499999999999,"5min":208.60499999999999,"15min":572.92100000000005},"last":6.9589999999999996},{"interface":"front","average":{"1min":19.026,"5min":14.109999999999999,"15min":21.210000000000001},"min":{"1min":6.8200000000000003,"5min":4.6980000000000004,"15min":4.0419999999999998},"max":{"1min":172.51499999999999,"5min":172.51499999999999,"15min":630.37900000000002},"last":19.815000000000001}]},{"osd":1,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.109000000000002,"5min":16.506,"15min":1146.9380000000001},"min":{"1min":7.5700000000000003,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":77.540000000000006,"5min":143.55699999999999,"15min":41558.328999999998},"last":56.890000000000001},{"interface":"front","average":{"1min":21.494,"5min":17.533999999999999,"15min":1148.0730000000001},"min":{"1min":7.633,"5min":6.8019999999999996,"15min":5.8220000000000001},"max":{"1min":176.839,"5min":176.839,"15min":41560.286999999997},"last":20.495999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":9.1359999999999992},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":56.274999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":25.748000000000001,"5min":22.646000000000001,"15min":24.829000000000001},"min":{"1min":9.4220000000000006,"5min":8.5069999999999997,"15min":6.4480000000000004},"max":{"1min":260.11700000000002,"5min":260.11700000000002,"15min":633.899},"last":55.115000000000002},{"interface":"front","average":{"1min":20.872,"5min":19.306999999999999,"15min":25.396999999999998},"min":{"1min":8.0060000000000002,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":177.78100000000001,"5min":188.101,"15min":582.32399999999996},"last":8.0109999999999992}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17750.601999999999,"5min":4460.9930000000004,"15min":1504.991},"min":{"1min":10.298,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":66604.820000000007,"5min":67109.214999999997,"15min":67109.214999999997},"last":21.734000000000002},{"interface":"front","average":{"1min":17580.571,"5min":4423.1670000000004,"15min":1494.211},"min":{"1min":8.0969999999999995,"5min":8.0969999999999995,"15min":8.0969999999999995},"max":{"1min":66274.171000000002,"5min":66779.236000000004,"15min":66779.236000000004},"last":9.875}]},{"osd":5,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":27.411999999999999,"5min":22.401,"15min":29.731000000000002},"min":{"1min":11.714,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":281.464,"5min":281.464,"15min":695.39700000000005},"last":54.112000000000002},{"interface":"front","average":{"1min":23.632999999999999,"5min":18.774000000000001,"15min":26.581},"min":{"1min":9.2300000000000004,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":222.58799999999999,"5min":222.58799999999999,"15min":577.46299999999997},"last":22.428999999999998}]},{"osd":7,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":25.933,"5min":22.887,"15min":22.504999999999999},"min":{"1min":10.718999999999999,"5min":10.291,"15min":10.291},"max":{"1min":246.16200000000001,"5min":246.16200000000001,"15min":246.16200000000001},"last":50.445999999999998},{"interface":"front","average":{"1min":21.619,"5min":20.536000000000001,"15min":20.823},"min":{"1min":8.0589999999999993,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":173.136,"5min":173.136,"15min":173.136},"last":10.507999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542604,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64220,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307620,"statfs":{"total":96636764160,"available":96571002880,"internally_reserved":0,"allocated":11292672,"data_stored":6716024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":6.3659999999999997},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":8.0619999999999994}]},{"osd":1,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.869,"5min":15.02,"15min":851.14999999999998},"min":{"1min":7.2220000000000004,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":14.837999999999999,"5min":216.709,"15min":42105.105000000003},"last":18.609000000000002},{"interface":"front","average":{"1min":13.176,"5min":19.975999999999999,"15min":856.74199999999996},"min":{"1min":7.9480000000000004,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":17.890000000000001,"5min":217.929,"15min":42107.510000000002},"last":23.559000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":23.015999999999998},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":19.792999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.619,"5min":17.259,"15min":15.612},"min":{"1min":5.8739999999999997,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":16.402000000000001,"5min":149.886,"15min":172.952},"last":17.966000000000001},{"interface":"front","average":{"1min":15.183,"5min":24.073,"15min":18.233000000000001},"min":{"1min":10.442,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":25.030000000000001,"5min":231.691,"15min":231.691},"last":26.625}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19513.460999999999,"5min":8370.6499999999996,"15min":2802.931},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":8.5519999999999996},"max":{"1min":66230.460000000006,"5min":66734.365999999995,"15min":66734.365999999995},"last":20.399000000000001},{"interface":"front","average":{"1min":19500.227999999999,"5min":8373.2870000000003,"15min":2802.7069999999999},"min":{"1min":8.0199999999999996,"5min":7.282,"15min":7.1840000000000002},"max":{"1min":66215.267999999996,"5min":66719.994999999995,"15min":66719.994999999995},"last":27.327999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.661,"5min":21.093,"15min":2609.1860000000001},"min":{"1min":7.8140000000000001,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":24.298999999999999,"5min":247.161,"15min":74994.611999999994},"last":25.129000000000001},{"interface":"front","average":{"1min":13.272,"5min":20.702999999999999,"15min":2611.3249999999998},"min":{"1min":8.0670000000000002,"5min":8.0670000000000002,"15min":8.0670000000000002},"max":{"1min":21.920000000000002,"5min":241.91,"15min":74995.471000000005},"last":7.4080000000000004}]},{"osd":7,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":15.432,"5min":20.175999999999998,"15min":23.984999999999999},"min":{"1min":11.771000000000001,"5min":9.0150000000000006,"15min":9.0150000000000006},"max":{"1min":23.018000000000001,"5min":112.215,"15min":112.215},"last":19.236000000000001},{"interface":"front","average":{"1min":15.993,"5min":20.603000000000002,"15min":26.225999999999999},"min":{"1min":11.492000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":22.132999999999999,"5min":103.886,"15min":103.886},"last":22.161999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542605,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6361429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":9.8640000000000008,"5min":21.134,"15min":14.946},"min":{"1min":7.0199999999999996,"5min":4.8760000000000003,"15min":4.5640000000000001},"max":{"1min":16.805,"5min":302.62700000000001,"15min":302.62700000000001},"last":11.864000000000001},{"interface":"front","average":{"1min":13.249000000000001,"5min":24.818000000000001,"15min":18.748000000000001},"min":{"1min":8.6639999999999997,"5min":5.9379999999999997,"15min":5.9379999999999997},"max":{"1min":18.440999999999999,"5min":289.18200000000002,"15min":289.18200000000002},"last":17.309999999999999}]},{"osd":1,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":8.0830000000000002,"5min":26.745999999999999,"15min":1846.819},"min":{"1min":5.8609999999999998,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":12.907,"5min":682.15599999999995,"15min":57448.739999999998},"last":5.5780000000000003},{"interface":"front","average":{"1min":12.319000000000001,"5min":33.448,"15min":1853.625},"min":{"1min":8.2409999999999997,"5min":7.4470000000000001,"15min":7.4169999999999998},"max":{"1min":16.358000000000001,"5min":685.01300000000003,"15min":57479.919999999998},"last":9.2010000000000005}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":6.5359999999999996},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":11.598000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.497999999999999,"5min":28.012,"15min":18.911000000000001},"min":{"1min":8.6999999999999993,"5min":6.048,"15min":6.048},"max":{"1min":17.545000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":7.5499999999999998},{"interface":"front","average":{"1min":11.528,"5min":26.763000000000002,"15min":18.524000000000001},"min":{"1min":8.1560000000000006,"5min":6.4699999999999998,"15min":6.4050000000000002},"max":{"1min":18.917999999999999,"5min":609.31899999999996,"15min":609.31899999999996},"last":13.407}]},{"osd":5,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":14.292999999999999,"5min":40.664999999999999,"15min":26.305},"min":{"1min":8.8930000000000007,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":35.802,"5min":607.48199999999997,"15min":607.48199999999997},"last":12.811},{"interface":"front","average":{"1min":13.429,"5min":27.794,"15min":21.152999999999999},"min":{"1min":8.0269999999999992,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":37.572000000000003,"5min":287.95100000000002,"15min":592.32100000000003},"last":14.058}]},{"osd":6,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.945,"5min":26.32,"15min":2144.3380000000002},"min":{"1min":7.0549999999999997,"5min":7.0549999999999997,"15min":6.9480000000000004},"max":{"1min":16.335999999999999,"5min":688.24300000000005,"15min":75975.607000000004},"last":8.4900000000000002},{"interface":"front","average":{"1min":13.26,"5min":27.960999999999999,"15min":2143.8119999999999},"min":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.018000000000001,"5min":610.89800000000002,"15min":75971.296000000002},"last":14.66}]},{"osd":7,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":142.12799999999999,"15min":67.069000000000003},"min":{"1min":12.108000000000001,"5min":10.411,"15min":10.411},"max":{"1min":17.981000000000002,"5min":611.59799999999996,"15min":611.59799999999996},"last":15.207000000000001},{"interface":"front","average":{"1min":13.507999999999999,"5min":140.16900000000001,"15min":57.863},"min":{"1min":8.8960000000000008,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":18.225000000000001,"5min":629.07500000000005,"15min":629.07500000000005},"last":15.843999999999999}]}]},{"osd":3,"up_from":736,"seq":3161095930305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103948,"kb_used_data":10628,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267892,"statfs":{"total":96636764160,"available":96530321408,"internally_reserved":0,"allocated":10883072,"data_stored":6333298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.074999999999999,"5min":13.201000000000001,"15min":14.895},"min":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":101.90600000000001,"5min":136.09999999999999,"15min":159.828},"last":41.670000000000002},{"interface":"front","average":{"1min":12.532,"5min":13.233000000000001,"15min":14.949},"min":{"1min":6.0149999999999997,"5min":4.4000000000000004,"15min":4.4000000000000004},"max":{"1min":43.442999999999998,"5min":153.459,"15min":153.459},"last":40.825000000000003}]},{"osd":1,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":8.8539999999999992,"5min":13.349,"15min":386.524},"min":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":13.83,"5min":82.028000000000006,"15min":20852.026000000002},"last":152.60900000000001},{"interface":"front","average":{"1min":12.772,"5min":15.637,"15min":382.35199999999998},"min":{"1min":9.5039999999999996,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":27.445,"5min":141.65000000000001,"15min":20733.266},"last":61.618000000000002}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":192.08600000000001},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":179.96700000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":13.74,"5min":21.388999999999999,"15min":22.102},"min":{"1min":8.6859999999999999,"5min":7.7439999999999998,"15min":7.0129999999999999},"max":{"1min":31.504000000000001,"5min":194.845,"15min":201.905},"last":202.30099999999999},{"interface":"front","average":{"1min":14.554,"5min":21.510000000000002,"15min":22.814},"min":{"1min":8.1989999999999998,"5min":8.1199999999999992,"15min":6.944},"max":{"1min":33.781999999999996,"5min":196.65600000000001,"15min":210.239},"last":179.37299999999999}]},{"osd":5,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":18.071999999999999,"5min":21.088000000000001,"15min":24.696000000000002},"min":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":86.810000000000002,"5min":177.97999999999999,"15min":290.07100000000003},"last":180.82499999999999},{"interface":"front","average":{"1min":17.045000000000002,"5min":17.774999999999999,"15min":20.707000000000001},"min":{"1min":9.657,"5min":7.6619999999999999,"15min":6.8220000000000001},"max":{"1min":72.721999999999994,"5min":143.268,"15min":268.577},"last":117.535}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":17.827999999999999,"5min":18.077999999999999,"15min":1643.2470000000001},"min":{"1min":11.323,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":36.439999999999998,"5min":94.352000000000004,"15min":76086.231},"last":98.412999999999997},{"interface":"front","average":{"1min":15.978999999999999,"5min":20.367000000000001,"15min":1652.1669999999999},"min":{"1min":9.4280000000000008,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":56.807000000000002,"5min":146.048,"15min":76087.114000000001},"last":62.909999999999997}]},{"osd":7,"last update":"Thu Nov 2 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":16.010000000000002,"5min":19.359000000000002,"15min":23.548999999999999},"min":{"1min":10.382,"5min":8.8870000000000005,"15min":8.8870000000000005},"max":{"1min":40.081000000000003,"5min":152.32599999999999,"15min":152.32599999999999},"last":178.74299999999999},{"interface":"front","average":{"1min":16.681999999999999,"5min":19.952000000000002,"15min":24.228000000000002},"min":{"1min":9.5419999999999998,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":39.018000000000001,"5min":140.87299999999999,"15min":140.87299999999999},"last":118.634}]}]},{"osd":7,"up_from":875,"seq":3758096384428,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63908,"kb_used_data":10716,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307932,"statfs":{"total":96636764160,"available":96571322368,"internally_reserved":0,"allocated":10973184,"data_stored":6414677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":10.130000000000001,"5min":13.368,"15min":15.576000000000001},"min":{"1min":6.1070000000000002,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":15.327,"5min":84.426000000000002,"15min":84.426000000000002},"last":10.09},{"interface":"front","average":{"1min":8.7460000000000004,"5min":13.207000000000001,"15min":18.576000000000001},"min":{"1min":5.8650000000000002,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":14.715999999999999,"5min":97.984999999999999,"15min":97.984999999999999},"last":8.0839999999999996}]},{"osd":1,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":12.630000000000001,"5min":16.503,"15min":22.439},"min":{"1min":7.5309999999999997,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":16.297000000000001,"5min":178.56,"15min":178.56},"last":14.006},{"interface":"front","average":{"1min":9.2569999999999997,"5min":12.856999999999999,"15min":15.263},"min":{"1min":6.6589999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":11.67,"5min":66.081999999999994,"15min":66.081999999999994},"last":6.6159999999999997}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":7.5030000000000001},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":13.164999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":14.497999999999999,"5min":20.501000000000001,"15min":19.623000000000001},"min":{"1min":8.9700000000000006,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":25.792000000000002,"5min":253.71000000000001,"15min":253.71000000000001},"last":53.985999999999997},{"interface":"front","average":{"1min":12.202,"5min":16.843,"15min":18.457999999999998},"min":{"1min":7.9500000000000002,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":17.898,"5min":101.316,"15min":101.316},"last":73.027000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:44 2023","interfaces":[{"interface":"back","average":{"1min":15907.763999999999,"5min":9828.0190000000002,"15min":3302.183},"min":{"1min":8.968,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":65773.679000000004,"5min":66277.987999999998,"15min":66277.987999999998},"last":15.157999999999999},{"interface":"front","average":{"1min":15924.536,"5min":9833.3089999999993,"15min":3301.9609999999998},"min":{"1min":11.398999999999999,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":65797.989000000001,"5min":66301.380000000005,"15min":66301.380000000005},"last":16.475000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":13.279,"5min":21.350999999999999,"15min":22.777999999999999},"min":{"1min":8.9740000000000002,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":17.827999999999999,"5min":263.65699999999998,"15min":263.65699999999998},"last":11.058},{"interface":"front","average":{"1min":14.351000000000001,"5min":22.829999999999998,"15min":24.738},"min":{"1min":9.6669999999999998,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":19.327999999999999,"5min":183.96899999999999,"15min":183.96899999999999},"last":25.789999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":18.141999999999999,"5min":24.789000000000001,"15min":26.550999999999998},"min":{"1min":10.255000000000001,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":72.542000000000002,"5min":285.56999999999999,"15min":285.56999999999999},"last":12.564},{"interface":"front","average":{"1min":17.75,"5min":22.82,"15min":23.931999999999999},"min":{"1min":9.7289999999999992,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":28.934000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":15.885999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:13:24.099 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-02T16:13:24.099 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:13:24.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.199+0000 7f5a7fe4c640 10 monclient: get_monmap_and_config 2023-11-02T16:13:24.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.199+0000 7f5a7fe4c640 10 monclient: build_initial_monmap 2023-11-02T16:13:24.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.199+0000 7f5a7fe4c640 10 monclient: monmap: 2023-11-02T16:13:24.202 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:24.202 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.202 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:24.200410+0000 2023-11-02T16:13:24.202 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:24.200410+0000 2023-11-02T16:13:24.202 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:24.203 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.206 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:24.206 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:24.206 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:24.206 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:24.207 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:24.207 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 Processor -- start 2023-11-02T16:13:24.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 -- start start 2023-11-02T16:13:24.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient: init 2023-11-02T16:13:24.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:24.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient: _add_conns ranks=[4,2,0,3,1] 2023-11-02T16:13:24.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5a780b4170 0x7f5a78141800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-c con 0x7f5a780b4170 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:24.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 0x7f5a78141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-b con 0x7f5a780a8df0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:24.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5a78142280 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:24.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780aa3d0 con 0x7f5a78142280 2023-11-02T16:13:24.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7813f420 con 0x7f5a780b4170 2023-11-02T16:13:24.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780b4550 con 0x7f5a780a8df0 2023-11-02T16:13:24.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:24.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.200+0000 7f5a7fe4c640 10 monclient(hunting): authenticate will time out at 3731.542597s 2023-11-02T16:13:24.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7d3e7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 0x7f5a78141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7e3e9640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f5a78142280 0x7f5a780b1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:56118/0 (socket says 172.21.15.174:56118) 2023-11-02T16:13:24.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7e3e9640 1 -- 172.21.15.174:0/2485824682 learned_addr learned my addr 172.21.15.174:0/2485824682 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:24.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7dbe8640 1 --2- 172.21.15.174:0/2485824682 >> v2:172.21.15.175:3300/0 conn(0x7f5a780b4170 0x7f5a78141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7d3e7640 10 monclient(hunting): get_auth_request con 0x7f5a780a8df0 auth_method 0 2023-11-02T16:13:24.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7d3e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7d3e7640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.201+0000 7f5a7d3e7640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): get_auth_request con 0x7f5a780b4170 auth_method 0 2023-11-02T16:13:24.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.202+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_done global_id 16895 payload 274 2023-11-02T16:13:24.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 1 -- 172.21.15.174:0/2485824682 >> v2:172.21.15.175:3300/0 conn(0x7f5a780b4170 msgr2=0x7f5a78141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:24.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 1 --2- 172.21.15.174:0/2485824682 >> v2:172.21.15.175:3300/0 conn(0x7f5a780b4170 0x7f5a78141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 1 -- 172.21.15.174:0/2485824682 >> v1:172.21.15.174:6789/0 conn(0x7f5a78142280 legacy=0x7f5a780b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 10 monclient: _finish_hunting 0 2023-11-02T16:13:24.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 1 monclient: found mon.noname-b 2023-11-02T16:13:24.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:24.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7cbe6640 1 -- 172.21.15.174:0/2485824682 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229286919 0 0) 0x7f5a780aa3d0 con 0x7f5a78142280 2023-11-02T16:13:24.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:24.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7d3e7640 1 -- 172.21.15.174:0/2485824682 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780a92d0 con 0x7f5a780a8df0 2023-11-02T16:13:24.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.204+0000 7f5a7cbe6640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-11-02T16:13:24.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.205+0000 7f5a7d3e7640 1 --2- 172.21.15.174:0/2485824682 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 0x7f5a78141d40 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f5a680097a0 tx=0x7f5a6802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dea82d666bf5ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:24.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 1 -- 172.21.15.174:0/2485824682 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6802e070 con 0x7f5a780a8df0 2023-11-02T16:13:24.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:24.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: dump: 2023-11-02T16:13:24.226 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.226 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.226 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.226 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.227 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: _finish_auth 0 2023-11-02T16:13:24.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: _check_auth_tickets 2023-11-02T16:13:24.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:24.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 1 -- 172.21.15.174:0/2485824682 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a680315a0 con 0x7f5a780a8df0 2023-11-02T16:13:24.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:24.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 1 -- 172.21.15.174:0/2485824682 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68031930 con 0x7f5a780a8df0 2023-11-02T16:13:24.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7fe4c640 5 monclient: authenticate success, global_id 16895 2023-11-02T16:13:24.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7fe4c640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:24.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7fe4c640 4 set_mon_vals no callback set 2023-11-02T16:13:24.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:24.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.209+0000 7f5a7cbe6640 10 monclient: dump: 2023-11-02T16:13:24.230 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.230 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.230 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.231 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.231 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.231 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.231 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.231 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 10 monclient: shutdown 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/2485824682 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 msgr2=0x7f5a78141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/2485824682 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 0x7f5a78141d40 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f5a680097a0 tx=0x7f5a6802b840 comp rx=0 tx=0).stop 2023-11-02T16:13:24.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/2485824682 shutdown_connections 2023-11-02T16:13:24.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/2485824682 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780a8df0 0x7f5a78141d40 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/2485824682 >> v2:172.21.15.175:3300/0 conn(0x7f5a780b4170 0x7f5a78141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/2485824682 >> 172.21.15.174:0/2485824682 conn(0x7f5a78065d80 msgr2=0x7f5a78066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:24.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/2485824682 shutdown_connections 2023-11-02T16:13:24.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.210+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/2485824682 wait complete. 2023-11-02T16:13:24.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 10 monclient: build_initial_monmap 2023-11-02T16:13:24.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 10 monclient: monmap: 2023-11-02T16:13:24.234 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:24.234 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:24.235 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:24.235 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:24.235 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:24.235 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.235 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 librados: starting msgr at 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 librados: starting objecter 2023-11-02T16:13:24.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 Processor -- start 2023-11-02T16:13:24.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 -- start start 2023-11-02T16:13:24.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 librados: setting wanted keys 2023-11-02T16:13:24.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 1 librados: calling monclient init 2023-11-02T16:13:24.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.211+0000 7f5a7fe4c640 10 monclient: init 2023-11-02T16:13:24.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:24.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:13:24.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 0x7f5a780ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-b con 0x7f5a780a8df0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:24.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 0x7f5a780abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-c con 0x7f5a780b4170 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a78142280 0x7f5a781da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): picked mon.noname-a con 0x7f5a78142280 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781d9c50 con 0x7f5a78142280 2023-11-02T16:13:24.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78145490 con 0x7f5a780a8df0 2023-11-02T16:13:24.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78143b00 con 0x7f5a780b4170 2023-11-02T16:13:24.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:24.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7fe4c640 10 monclient(hunting): authenticate will time out at 3731.553751s 2023-11-02T16:13:24.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 0x7f5a780abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7e3e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a78142280 0x7f5a781da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7dbe8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 0x7f5a780ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 0x7f5a780abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46348/0 (socket says 172.21.15.174:46348) 2023-11-02T16:13:24.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 1 -- 172.21.15.174:0/3116680545 learned_addr learned my addr 172.21.15.174:0/3116680545 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:24.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 10 monclient(hunting): get_auth_request con 0x7f5a780b4170 auth_method 0 2023-11-02T16:13:24.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.212+0000 7f5a7d3e7640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): get_auth_request con 0x7f5a78142280 auth_method 0 2023-11-02T16:13:24.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): get_auth_request con 0x7f5a780a8df0 auth_method 0 2023-11-02T16:13:24.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7d3e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7e3e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.213+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 10 monclient(hunting): handle_auth_done global_id 21496 payload 824 2023-11-02T16:13:24.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 -- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 msgr2=0x7f5a780abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 0x7f5a780abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 -- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a78142280 msgr2=0x7f5a781da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a78142280 0x7f5a781da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 10 monclient: _finish_hunting 0 2023-11-02T16:13:24.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 monclient: found mon.noname-b 2023-11-02T16:13:24.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:24.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.216+0000 7f5a7dbe8640 1 -- 172.21.15.174:0/3116680545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009400 con 0x7f5a780a8df0 2023-11-02T16:13:24.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.218+0000 7f5a7dbe8640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 0x7f5a780ab280 secure :-1 s=READY pgs=5256 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e850 tx=0x7f5a7400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:24.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.218+0000 7f5a6e7fc640 10 client.?.objecter ms_handle_connect 0x7f5a780a8df0 2023-11-02T16:13:24.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.218+0000 7f5a6e7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:24.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a740095b0 con 0x7f5a780a8df0 2023-11-02T16:13:24.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:24.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: dump: 2023-11-02T16:13:24.250 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.251 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.251 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.251 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.251 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.251 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.252 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.252 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.252 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.252 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:24.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:24.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:24.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7400c940 con 0x7f5a780a8df0 2023-11-02T16:13:24.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:24.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400ccd0 con 0x7f5a780a8df0 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 5 monclient: authenticate success, global_id 21496 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 10 monclient: _renew_subs 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a780ac290 con 0x7f5a780a8df0 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6ffff640 4 set_mon_vals no callback set 2023-11-02T16:13:24.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:24.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a6e7fc640 10 monclient: dump: 2023-11-02T16:13:24.255 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.255 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.255 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.256 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.256 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.256 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.256 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.256 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.257 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.257 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 10 client.21496.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:24.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 10 monclient: _renew_subs 2023-11-02T16:13:24.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781da800 con 0x7f5a780a8df0 2023-11-02T16:13:24.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.222+0000 7f5a7fe4c640 1 librados: init done 2023-11-02T16:13:24.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.223+0000 7f5a7fe4c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:24.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.223+0000 7f5a7fe4c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:24.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.223+0000 7f5a7fe4c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.223+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a781daa50 con 0x7f5a780a8df0 2023-11-02T16:13:24.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.462+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5a74013470 con 0x7f5a780a8df0 2023-11-02T16:13:24.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.462+0000 7f5a6e7fc640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 0x7f5a5c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.463+0000 7f5a7d3e7640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 0x7f5a5c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.463+0000 7f5a7d3e7640 10 monclient: get_auth_request con 0x7f5a5c06be10 auth_method 0 2023-11-02T16:13:24.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.463+0000 7f5a6e7fc640 10 client.21496.objecter ms_handle_connect 0x7f5a5c06be10 2023-11-02T16:13:24.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.463+0000 7f5a7d3e7640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 0x7f5a5c06e2d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5a680097a0 tx=0x7f5a6803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:24.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.466+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1029..1029 src has 247..1029) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f5a7400ce30 con 0x7f5a780a8df0 2023-11-02T16:13:24.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.466+0000 7f5a6e7fc640 10 client.21496.objecter ms_dispatch 0x7f5a78000b90 osd_map(1029..1029 src has 247..1029) v4 2023-11-02T16:13:24.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.466+0000 7f5a6e7fc640 3 client.21496.objecter handle_osd_map got epochs [1029,1029] > 0 2023-11-02T16:13:24.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.466+0000 7f5a6e7fc640 3 client.21496.objecter handle_osd_map decoding full epoch 1029 2023-11-02T16:13:24.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.466+0000 7f5a6e7fc640 20 client.21496.objecter dump_active .. 0 homeless 2023-11-02T16:13:24.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.475+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5a740546d0 con 0x7f5a780a8df0 2023-11-02T16:13:24.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.475+0000 7f5a6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:24.477 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.475+0000 7f5a6e7fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:24.684 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.682+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a781daa50 con 0x7f5a5c06be10 2023-11-02T16:13:24.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.686+0000 7f5a6e7fc640 1 -- 172.21.15.174:0/3116680545 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152030 (secure 0 0 0) 0x7f5a781daa50 con 0x7f5a5c06be10 2023-11-02T16:13:24.690 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:24.690 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:13:24.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 10 librados: watch_flush enter 2023-11-02T16:13:24.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 10 librados: watch_flush exit 2023-11-02T16:13:24.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 20 client.21496.objecter shutdown clearing up homeless session... 2023-11-02T16:13:24.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 10 client.21496.objecter successfully canceled tick 2023-11-02T16:13:24.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 msgr2=0x7f5a5c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 0x7f5a5c06e2d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5a680097a0 tx=0x7f5a6803d040 comp rx=0 tx=0).stop 2023-11-02T16:13:24.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 10 monclient: shutdown 2023-11-02T16:13:24.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:24.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 msgr2=0x7f5a780ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 0x7f5a780ab280 secure :-1 s=READY pgs=5256 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e850 tx=0x7f5a7400ed10 comp rx=0 tx=0).stop 2023-11-02T16:13:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 shutdown_connections 2023-11-02T16:13:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a78142280 0x7f5a781da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f5a5c06be10 0x7f5a5c06e2d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a780b4170 0x7f5a780abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 --2- 172.21.15.174:0/3116680545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a780a8df0 0x7f5a780ab280 unknown :-1 s=CLOSED pgs=5256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 >> 172.21.15.174:0/3116680545 conn(0x7f5a78065d80 msgr2=0x7f5a780b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:24.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 shutdown_connections 2023-11-02T16:13:24.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 -- 172.21.15.174:0/3116680545 wait complete. 2023-11-02T16:13:24.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.689+0000 7f5a7fe4c640 1 librados: shutdown 2023-11-02T16:13:24.702 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2219,"stamp":"2023-11-02T16:13:24.568191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609840,"kb_used_data":86000,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364880,"statfs":{"total":773094113280,"available":772469637120,"internally_reserved":0,"allocated":88064000,"data_stored":51514925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":417,"apply_latency_ms":417,"commit_latency_ns":417000000,"apply_latency_ns":417000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4000299,"num_objects":-44,"num_object_clones":-1,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-44,"num_whiteouts":0,"num_read":-31,"num_read_kb":-11,"num_write":-71,"num_write_kb":-3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.165371"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":998,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.956626+0000","last_change":"2023-11-02T16:13:17.157671+0000","last_active":"2023-11-02T16:13:17.956626+0000","last_peered":"2023-11-02T16:13:17.956626+0000","last_clean":"2023-11-02T16:13:17.956626+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:17.956626+0000","last_undegraded":"2023-11-02T16:13:17.956626+0000","last_fullsized":"2023-11-02T16:13:17.956626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:58:17.271177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.749561+0000","last_change":"2023-11-02T16:07:27.307241+0000","last_active":"2023-11-02T16:13:17.749561+0000","last_peered":"2023-11-02T16:13:17.749561+0000","last_clean":"2023-11-02T16:13:17.749561+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:17.749561+0000","last_undegraded":"2023-11-02T16:13:17.749561+0000","last_fullsized":"2023-11-02T16:13:17.749561+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:12:49.486828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.186771+0000","last_change":"2023-11-02T16:07:27.395825+0000","last_active":"2023-11-02T16:13:21.186771+0000","last_peered":"2023-11-02T16:13:21.186771+0000","last_clean":"2023-11-02T16:13:21.186771+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:21.186771+0000","last_undegraded":"2023-11-02T16:13:21.186771+0000","last_fullsized":"2023-11-02T16:13:21.186771+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:25:43.494367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":972,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.121930+0000","last_change":"2023-11-02T15:35:17.345372+0000","last_active":"2023-11-02T16:13:19.121930+0000","last_peered":"2023-11-02T16:13:19.121930+0000","last_clean":"2023-11-02T16:13:19.121930+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:19.121930+0000","last_undegraded":"2023-11-02T16:13:19.121930+0000","last_fullsized":"2023-11-02T16:13:19.121930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:39:06.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.159925+0000","last_change":"2023-11-02T15:35:14.211126+0000","last_active":"2023-11-02T16:13:21.159925+0000","last_peered":"2023-11-02T16:13:21.159925+0000","last_clean":"2023-11-02T16:13:21.159925+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:21.159925+0000","last_undegraded":"2023-11-02T16:13:21.159925+0000","last_fullsized":"2023-11-02T16:13:21.159925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:19:16.985396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":920,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.171904+0000","last_change":"2023-11-02T15:59:17.698985+0000","last_active":"2023-11-02T16:13:21.171904+0000","last_peered":"2023-11-02T16:13:21.171904+0000","last_clean":"2023-11-02T16:13:21.171904+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:21.171904+0000","last_undegraded":"2023-11-02T16:13:21.171904+0000","last_fullsized":"2023-11-02T16:13:21.171904+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:31:01.071757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":998,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.146603+0000","last_change":"2023-11-02T16:13:16.820908+0000","last_active":"2023-11-02T16:13:21.146603+0000","last_peered":"2023-11-02T16:13:21.146603+0000","last_clean":"2023-11-02T16:13:21.146603+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:21.146603+0000","last_undegraded":"2023-11-02T16:13:21.146603+0000","last_fullsized":"2023-11-02T16:13:21.146603+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:08:51.272032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":972,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.143826+0000","last_change":"2023-11-02T15:35:30.105402+0000","last_active":"2023-11-02T16:13:21.143826+0000","last_peered":"2023-11-02T16:13:21.143826+0000","last_clean":"2023-11-02T16:13:21.143826+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:21.143826+0000","last_undegraded":"2023-11-02T16:13:21.143826+0000","last_fullsized":"2023-11-02T16:13:21.143826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:00:16.092163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":960,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.061767+0000","last_change":"2023-11-02T16:07:27.407922+0000","last_active":"2023-11-02T16:13:20.061767+0000","last_peered":"2023-11-02T16:13:20.061767+0000","last_clean":"2023-11-02T16:13:20.061767+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:20.061767+0000","last_undegraded":"2023-11-02T16:13:20.061767+0000","last_fullsized":"2023-11-02T16:13:20.061767+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:16:50.921541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1001,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.087417+0000","last_change":"2023-11-02T16:13:16.885325+0000","last_active":"2023-11-02T16:13:20.087417+0000","last_peered":"2023-11-02T16:13:20.087417+0000","last_clean":"2023-11-02T16:13:20.087417+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:20.087417+0000","last_undegraded":"2023-11-02T16:13:20.087417+0000","last_fullsized":"2023-11-02T16:13:20.087417+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:18:37.960941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1011,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.069466+0000","last_change":"2023-11-02T16:13:16.913467+0000","last_active":"2023-11-02T16:13:20.069466+0000","last_peered":"2023-11-02T16:13:20.069466+0000","last_clean":"2023-11-02T16:13:20.069466+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:20.069466+0000","last_undegraded":"2023-11-02T16:13:20.069466+0000","last_fullsized":"2023-11-02T16:13:20.069466+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:05:41.639058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1008,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.719637+0000","last_change":"2023-11-02T16:13:17.189587+0000","last_active":"2023-11-02T16:13:17.719637+0000","last_peered":"2023-11-02T16:13:17.719637+0000","last_clean":"2023-11-02T16:13:17.719637+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:17.719637+0000","last_undegraded":"2023-11-02T16:13:17.719637+0000","last_fullsized":"2023-11-02T16:13:17.719637+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:04:14.846503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":983,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.746564+0000","last_change":"2023-11-02T15:59:17.640186+0000","last_active":"2023-11-02T16:13:17.746564+0000","last_peered":"2023-11-02T16:13:17.746564+0000","last_clean":"2023-11-02T16:13:17.746564+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:17.746564+0000","last_undegraded":"2023-11-02T16:13:17.746564+0000","last_fullsized":"2023-11-02T16:13:17.746564+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:57.212864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.744965+0000","last_change":"2023-11-02T15:59:17.577430+0000","last_active":"2023-11-02T16:13:17.744965+0000","last_peered":"2023-11-02T16:13:17.744965+0000","last_clean":"2023-11-02T16:13:17.744965+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:17.744965+0000","last_undegraded":"2023-11-02T16:13:17.744965+0000","last_fullsized":"2023-11-02T16:13:17.744965+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:03:26.273290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":973,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.959279+0000","last_change":"2023-11-02T15:35:23.837202+0000","last_active":"2023-11-02T16:13:17.959279+0000","last_peered":"2023-11-02T16:13:17.959279+0000","last_clean":"2023-11-02T16:13:17.959279+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:17.959279+0000","last_undegraded":"2023-11-02T16:13:17.959279+0000","last_fullsized":"2023-11-02T16:13:17.959279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:03:24.083855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":978,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.094447+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T16:13:19.094447+0000","last_peered":"2023-11-02T16:13:19.094447+0000","last_clean":"2023-11-02T16:13:19.094447+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:19.094447+0000","last_undegraded":"2023-11-02T16:13:19.094447+0000","last_fullsized":"2023-11-02T16:13:19.094447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":987,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.954242+0000","last_change":"2023-11-02T15:59:17.883690+0000","last_active":"2023-11-02T16:13:17.954242+0000","last_peered":"2023-11-02T16:13:17.954242+0000","last_clean":"2023-11-02T16:13:17.954242+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:17.954242+0000","last_undegraded":"2023-11-02T16:13:17.954242+0000","last_fullsized":"2023-11-02T16:13:17.954242+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:58:33.336681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":989,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.718156+0000","last_change":"2023-11-02T15:59:17.612075+0000","last_active":"2023-11-02T16:13:17.718156+0000","last_peered":"2023-11-02T16:13:17.718156+0000","last_clean":"2023-11-02T16:13:17.718156+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:17.718156+0000","last_undegraded":"2023-11-02T16:13:17.718156+0000","last_fullsized":"2023-11-02T16:13:17.718156+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:08:04.600098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1013,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.071639+0000","last_change":"2023-11-02T16:13:17.151154+0000","last_active":"2023-11-02T16:13:20.071639+0000","last_peered":"2023-11-02T16:13:20.071639+0000","last_clean":"2023-11-02T16:13:20.071639+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:20.071639+0000","last_undegraded":"2023-11-02T16:13:20.071639+0000","last_fullsized":"2023-11-02T16:13:20.071639+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:40.667953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":952,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.080989+0000","last_change":"2023-11-02T15:43:56.978148+0000","last_active":"2023-11-02T16:13:20.080989+0000","last_peered":"2023-11-02T16:13:20.080989+0000","last_clean":"2023-11-02T16:13:20.080989+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:20.080989+0000","last_undegraded":"2023-11-02T16:13:20.080989+0000","last_fullsized":"2023-11-02T16:13:20.080989+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:17:20.801998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":533,"reported_epoch":1028,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.083506+0000","last_change":"2023-11-02T16:07:27.487367+0000","last_active":"2023-11-02T16:13:22.083506+0000","last_peered":"2023-11-02T16:13:22.083506+0000","last_clean":"2023-11-02T16:13:22.083506+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:22.083506+0000","last_undegraded":"2023-11-02T16:13:22.083506+0000","last_fullsized":"2023-11-02T16:13:22.083506+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:15:14.310785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1004,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.696771+0000","last_change":"2023-11-02T15:59:17.535951+0000","last_active":"2023-11-02T16:13:17.696771+0000","last_peered":"2023-11-02T16:13:17.696771+0000","last_clean":"2023-11-02T16:13:17.696771+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:17.696771+0000","last_undegraded":"2023-11-02T16:13:17.696771+0000","last_fullsized":"2023-11-02T16:13:17.696771+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:50:51.454223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":975,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.934494+0000","last_change":"2023-11-02T15:35:16.434956+0000","last_active":"2023-11-02T16:13:17.934494+0000","last_peered":"2023-11-02T16:13:17.934494+0000","last_clean":"2023-11-02T16:13:17.934494+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:17.934494+0000","last_undegraded":"2023-11-02T16:13:17.934494+0000","last_fullsized":"2023-11-02T16:13:17.934494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:54:41.093040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":989,"reported_epoch":1028,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.081915+0000","last_change":"2023-11-02T16:07:27.307462+0000","last_active":"2023-11-02T16:13:22.081915+0000","last_peered":"2023-11-02T16:13:22.081915+0000","last_clean":"2023-11-02T16:13:22.081915+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:22.081915+0000","last_undegraded":"2023-11-02T16:13:22.081915+0000","last_fullsized":"2023-11-02T16:13:22.081915+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:53:09.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":935,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.988390+0000","last_change":"2023-11-02T15:35:10.705051+0000","last_active":"2023-11-02T16:13:19.988390+0000","last_peered":"2023-11-02T16:13:19.988390+0000","last_clean":"2023-11-02T16:13:19.988390+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:19.988390+0000","last_undegraded":"2023-11-02T16:13:19.988390+0000","last_fullsized":"2023-11-02T16:13:19.988390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:36:19.002698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":990,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.703806+0000","last_change":"2023-11-02T15:59:17.407544+0000","last_active":"2023-11-02T16:13:17.703806+0000","last_peered":"2023-11-02T16:13:17.703806+0000","last_clean":"2023-11-02T16:13:17.703806+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:17.703806+0000","last_undegraded":"2023-11-02T16:13:17.703806+0000","last_fullsized":"2023-11-02T16:13:17.703806+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:52:17.493086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":973,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.156486+0000","last_change":"2023-11-02T15:35:30.088014+0000","last_active":"2023-11-02T16:13:21.156486+0000","last_peered":"2023-11-02T16:13:21.156486+0000","last_clean":"2023-11-02T16:13:21.156486+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:21.156486+0000","last_undegraded":"2023-11-02T16:13:21.156486+0000","last_fullsized":"2023-11-02T16:13:21.156486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:24:59.243202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.710169+0000","last_change":"2023-11-02T16:13:17.216261+0000","last_active":"2023-11-02T16:13:17.710169+0000","last_peered":"2023-11-02T16:13:17.710169+0000","last_clean":"2023-11-02T16:13:17.710169+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:17.710169+0000","last_undegraded":"2023-11-02T16:13:17.710169+0000","last_fullsized":"2023-11-02T16:13:17.710169+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:51:41.117006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"22'76","reported_seq":1085,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.149378+0000","last_change":"2023-11-02T15:33:55.693910+0000","last_active":"2023-11-02T16:13:21.149378+0000","last_peered":"2023-11-02T16:13:21.149378+0000","last_clean":"2023-11-02T16:13:21.149378+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:21.149378+0000","last_undegraded":"2023-11-02T16:13:21.149378+0000","last_fullsized":"2023-11-02T16:13:21.149378+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:33:53.161912+0000","last_clean_scrub_stamp":"2023-11-02T15:33:53.161912+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:29.329385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"40'7","reported_seq":1015,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.936228+0000","last_change":"2023-11-02T16:13:17.029849+0000","last_active":"2023-11-02T16:13:17.936228+0000","last_peered":"2023-11-02T16:13:17.936228+0000","last_clean":"2023-11-02T16:13:17.936228+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:17.936228+0000","last_undegraded":"2023-11-02T16:13:17.936228+0000","last_fullsized":"2023-11-02T16:13:17.936228+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:45:56.714871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1002,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.038464+0000","last_change":"2023-11-02T16:13:17.007381+0000","last_active":"2023-11-02T16:13:20.038464+0000","last_peered":"2023-11-02T16:13:20.038464+0000","last_clean":"2023-11-02T16:13:20.038464+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:20.038464+0000","last_undegraded":"2023-11-02T16:13:20.038464+0000","last_fullsized":"2023-11-02T16:13:20.038464+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:29:31.495127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1002,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.079106+0000","last_change":"2023-11-02T16:13:16.951026+0000","last_active":"2023-11-02T16:13:19.079106+0000","last_peered":"2023-11-02T16:13:19.079106+0000","last_clean":"2023-11-02T16:13:19.079106+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:19.079106+0000","last_undegraded":"2023-11-02T16:13:19.079106+0000","last_fullsized":"2023-11-02T16:13:19.079106+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:01:56.841900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":962,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:20.028826+0000","last_change":"2023-11-02T16:13:17.159877+0000","last_active":"2023-11-02T16:13:20.028826+0000","last_peered":"2023-11-02T16:13:20.028826+0000","last_clean":"2023-11-02T16:13:20.028826+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:20.028826+0000","last_undegraded":"2023-11-02T16:13:20.028826+0000","last_fullsized":"2023-11-02T16:13:20.028826+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:32:03.253445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":987,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.120389+0000","last_change":"2023-11-02T15:43:57.129812+0000","last_active":"2023-11-02T16:13:19.120389+0000","last_peered":"2023-11-02T16:13:19.120389+0000","last_clean":"2023-11-02T16:13:19.120389+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:19.120389+0000","last_undegraded":"2023-11-02T16:13:19.120389+0000","last_fullsized":"2023-11-02T16:13:19.120389+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:42.294660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":976,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.092801+0000","last_change":"2023-11-02T15:35:17.386786+0000","last_active":"2023-11-02T16:13:19.092801+0000","last_peered":"2023-11-02T16:13:19.092801+0000","last_clean":"2023-11-02T16:13:19.092801+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:19.092801+0000","last_undegraded":"2023-11-02T16:13:19.092801+0000","last_fullsized":"2023-11-02T16:13:19.092801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:58:39.849736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":984,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.748036+0000","last_change":"2023-11-02T15:59:17.643298+0000","last_active":"2023-11-02T16:13:17.748036+0000","last_peered":"2023-11-02T16:13:17.748036+0000","last_clean":"2023-11-02T16:13:17.748036+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:17.748036+0000","last_undegraded":"2023-11-02T16:13:17.748036+0000","last_fullsized":"2023-11-02T16:13:17.748036+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:31:03.424299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":985,"reported_epoch":1025,"state":"active+clean","last_fresh":"2023-11-02T16:13:17.725844+0000","last_change":"2023-11-02T15:59:17.471575+0000","last_active":"2023-11-02T16:13:17.725844+0000","last_peered":"2023-11-02T16:13:17.725844+0000","last_clean":"2023-11-02T16:13:17.725844+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:17.725844+0000","last_undegraded":"2023-11-02T16:13:17.725844+0000","last_fullsized":"2023-11-02T16:13:17.725844+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:57:21.803164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":933,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.183490+0000","last_change":"2023-11-02T16:13:16.821060+0000","last_active":"2023-11-02T16:13:21.183490+0000","last_peered":"2023-11-02T16:13:21.183490+0000","last_clean":"2023-11-02T16:13:21.183490+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:21.183490+0000","last_undegraded":"2023-11-02T16:13:21.183490+0000","last_fullsized":"2023-11-02T16:13:21.183490+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:46.475704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.055317+0000","last_change":"2023-11-02T16:07:27.399974+0000","last_active":"2023-11-02T16:13:19.055317+0000","last_peered":"2023-11-02T16:13:19.055317+0000","last_clean":"2023-11-02T16:13:19.055317+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:19.055317+0000","last_undegraded":"2023-11-02T16:13:19.055317+0000","last_fullsized":"2023-11-02T16:13:19.055317+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:02:28.668857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1000,"reported_epoch":1026,"state":"active+clean","last_fresh":"2023-11-02T16:13:19.077626+0000","last_change":"2023-11-02T16:13:16.955603+0000","last_active":"2023-11-02T16:13:19.077626+0000","last_peered":"2023-11-02T16:13:19.077626+0000","last_clean":"2023-11-02T16:13:19.077626+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:19.077626+0000","last_undegraded":"2023-11-02T16:13:19.077626+0000","last_fullsized":"2023-11-02T16:13:19.077626+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:12:56.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":997,"reported_epoch":1027,"state":"active+clean","last_fresh":"2023-11-02T16:13:21.158342+0000","last_change":"2023-11-02T16:13:16.834330+0000","last_active":"2023-11-02T16:13:21.158342+0000","last_peered":"2023-11-02T16:13:21.158342+0000","last_clean":"2023-11-02T16:13:21.158342+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:21.158342+0000","last_undegraded":"2023-11-02T16:13:21.158342+0000","last_fullsized":"2023-11-02T16:13:21.158342+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T16:56:31.478240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544251,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63832,"kb_used_data":10640,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308008,"statfs":{"total":96636764160,"available":96571400192,"internally_reserved":0,"allocated":10895360,"data_stored":6318201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":15.131},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":6.7720000000000002}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":9.5830000000000002},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":13.759}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":10.568},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":8.0370000000000008}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":16.782},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":18.870000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":11.778},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":14.539999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":15.752000000000001},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":11.177}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":18.274999999999999},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":12.448}]}]},{"osd":1,"up_from":16,"seq":68719477187,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104348,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267492,"statfs":{"total":96636764160,"available":96529911808,"internally_reserved":0,"allocated":11292672,"data_stored":6716036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":8.7509999999999994,"5min":11.885,"15min":17.582000000000001},"min":{"1min":4.5650000000000004,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":13.523999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":13.021000000000001},{"interface":"front","average":{"1min":8.1389999999999993,"5min":12.238,"15min":18.489999999999998},"min":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":11.058999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":13.617000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":28.122},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":27.352}]},{"osd":3,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":11.907,"5min":13.795999999999999,"15min":21.059999999999999},"min":{"1min":8.5370000000000008,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":26.161000000000001,"5min":77.778000000000006,"15min":549.92200000000003},"last":11.098000000000001},{"interface":"front","average":{"1min":12.789999999999999,"5min":17.585999999999999,"15min":24.571000000000002},"min":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":7.8280000000000003},"max":{"1min":26.827000000000002,"5min":108.244,"15min":542.60500000000002},"last":7.3369999999999997}]},{"osd":4,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":107.46299999999999,"5min":110.161,"15min":49.548000000000002},"min":{"1min":7.8819999999999997,"5min":7.8819999999999997,"15min":6.1360000000000001},"max":{"1min":1884.9880000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":9.391},{"interface":"front","average":{"1min":111.589,"5min":110.36199999999999,"15min":51.369},"min":{"1min":7.6950000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"max":{"1min":1970.0229999999999,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":8.8599999999999994}]},{"osd":5,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":16.436,"15min":25.068999999999999},"min":{"1min":8.0990000000000002,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":16.541,"5min":71.225999999999999,"15min":529.93499999999995},"last":36.536999999999999},{"interface":"front","average":{"1min":15.509,"5min":19.974,"15min":26.859000000000002},"min":{"1min":9.5839999999999996,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":23.559999999999999,"5min":122.14,"15min":555.77700000000004},"last":29.494}]},{"osd":6,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":15.324999999999999,"5min":22.655000000000001,"15min":1553.2819999999999},"min":{"1min":9.0719999999999992,"5min":8.6210000000000004,"15min":8.1400000000000006},"max":{"1min":31.744,"5min":245.36500000000001,"15min":73455.679000000004},"last":14.265000000000001},{"interface":"front","average":{"1min":15.226000000000001,"5min":17.460999999999999,"15min":1549.6890000000001},"min":{"1min":9.5269999999999992,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":30.879999999999999,"5min":124.36499999999999,"15min":73453.190000000002},"last":11.747999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:12:35 2023","interfaces":[{"interface":"back","average":{"1min":15.096,"5min":18.516999999999999,"15min":21.047000000000001},"min":{"1min":10.757999999999999,"5min":10.097,"15min":10.097},"max":{"1min":21.617999999999999,"5min":91.019999999999996,"15min":91.019999999999996},"last":9.9969999999999999},{"interface":"front","average":{"1min":16.209,"5min":22.247,"15min":24.065999999999999},"min":{"1min":10.824999999999999,"5min":10.113,"15min":10.113},"max":{"1min":28.861999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":15.321999999999999}]}]},{"osd":0,"up_from":16,"seq":68719477211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81968,"kb_used_data":10728,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289872,"statfs":{"total":96636764160,"available":96552828928,"internally_reserved":0,"allocated":10985472,"data_stored":6426109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:12:55 2023","interfaces":[{"interface":"back","average":{"1min":8.7309999999999999,"5min":13.488,"15min":77.375},"min":{"1min":5.3490000000000002,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.488,"5min":107.59999999999999,"15min":5781.2129999999997},"last":7.5629999999999997},{"interface":"front","average":{"1min":8.4719999999999995,"5min":11.718,"15min":74.570999999999998},"min":{"1min":6.2249999999999996,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":13.672000000000001,"5min":73.094999999999999,"15min":5443.4889999999996},"last":11.455}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.778},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":13.651999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":12.1,"5min":17.452999999999999,"15min":17.625},"min":{"1min":8.3399999999999999,"5min":7.46,"15min":7.46},"max":{"1min":29.192,"5min":156.99100000000001,"15min":234.29300000000001},"last":40.524999999999999},{"interface":"front","average":{"1min":12.449999999999999,"5min":17.09,"15min":15.265000000000001},"min":{"1min":7.6239999999999997,"5min":7.4950000000000001,"15min":6.6070000000000002},"max":{"1min":34.267000000000003,"5min":151.30099999999999,"15min":151.30099999999999},"last":62.927}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":21213.947,"5min":17485.800999999999,"15min":5840.5320000000002},"min":{"1min":9.7330000000000005,"5min":7.1600000000000001,"15min":6.3200000000000003},"max":{"1min":64456.964,"5min":66161.077000000005,"15min":66161.077000000005},"last":14.664},{"interface":"front","average":{"1min":21222.184000000001,"5min":17490.220000000001,"15min":5843.1549999999997},"min":{"1min":11.762,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":64460.497000000003,"5min":66163.926000000007,"15min":66163.926000000007},"last":15.914}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":16.919},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":17.655000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":18.556999999999999,"5min":22.667000000000002,"15min":1772.4739999999999},"min":{"1min":14.353999999999999,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":38.277000000000001,"5min":107.51000000000001,"15min":71055.725999999995},"last":18.425000000000001},{"interface":"front","average":{"1min":13.893000000000001,"5min":19.393000000000001,"15min":1770.963},"min":{"1min":7.7320000000000002,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":32.561,"5min":156.91,"15min":71055.100000000006},"last":15.398}]},{"osd":7,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19.140000000000001,"5min":25.041,"15min":30.241},"min":{"1min":13.130000000000001,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":44.636000000000003,"5min":205.88999999999999,"15min":205.88999999999999},"last":19.449000000000002},{"interface":"front","average":{"1min":11.23,"5min":17.503,"15min":22.859000000000002},"min":{"1min":8.0190000000000001,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":31.449999999999999,"5min":155.96600000000001,"15min":155.96600000000001},"last":12.237}]}]},{"osd":6,"up_from":387,"seq":1662152343954,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63828,"kb_used_data":10636,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308012,"statfs":{"total":96636764160,"available":96571404288,"internally_reserved":0,"allocated":10891264,"data_stored":6318196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.405000000000001,"5min":14.585000000000001,"15min":20.376999999999999},"min":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.218},"max":{"1min":208.60499999999999,"5min":208.60499999999999,"15min":572.92100000000005},"last":6.9589999999999996},{"interface":"front","average":{"1min":19.026,"5min":14.109999999999999,"15min":21.210000000000001},"min":{"1min":6.8200000000000003,"5min":4.6980000000000004,"15min":4.0419999999999998},"max":{"1min":172.51499999999999,"5min":172.51499999999999,"15min":630.37900000000002},"last":19.815000000000001}]},{"osd":1,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.109000000000002,"5min":16.506,"15min":1146.9380000000001},"min":{"1min":7.5700000000000003,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":77.540000000000006,"5min":143.55699999999999,"15min":41558.328999999998},"last":56.890000000000001},{"interface":"front","average":{"1min":21.494,"5min":17.533999999999999,"15min":1148.0730000000001},"min":{"1min":7.633,"5min":6.8019999999999996,"15min":5.8220000000000001},"max":{"1min":176.839,"5min":176.839,"15min":41560.286999999997},"last":20.495999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":9.1359999999999992},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":56.274999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":25.748000000000001,"5min":22.646000000000001,"15min":24.829000000000001},"min":{"1min":9.4220000000000006,"5min":8.5069999999999997,"15min":6.4480000000000004},"max":{"1min":260.11700000000002,"5min":260.11700000000002,"15min":633.899},"last":55.115000000000002},{"interface":"front","average":{"1min":20.872,"5min":19.306999999999999,"15min":25.396999999999998},"min":{"1min":8.0060000000000002,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":177.78100000000001,"5min":188.101,"15min":582.32399999999996},"last":8.0109999999999992}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17750.601999999999,"5min":4460.9930000000004,"15min":1504.991},"min":{"1min":10.298,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":66604.820000000007,"5min":67109.214999999997,"15min":67109.214999999997},"last":21.734000000000002},{"interface":"front","average":{"1min":17580.571,"5min":4423.1670000000004,"15min":1494.211},"min":{"1min":8.0969999999999995,"5min":8.0969999999999995,"15min":8.0969999999999995},"max":{"1min":66274.171000000002,"5min":66779.236000000004,"15min":66779.236000000004},"last":9.875}]},{"osd":5,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":27.411999999999999,"5min":22.401,"15min":29.731000000000002},"min":{"1min":11.714,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":281.464,"5min":281.464,"15min":695.39700000000005},"last":54.112000000000002},{"interface":"front","average":{"1min":23.632999999999999,"5min":18.774000000000001,"15min":26.581},"min":{"1min":9.2300000000000004,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":222.58799999999999,"5min":222.58799999999999,"15min":577.46299999999997},"last":22.428999999999998}]},{"osd":7,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":25.933,"5min":22.887,"15min":22.504999999999999},"min":{"1min":10.718999999999999,"5min":10.291,"15min":10.291},"max":{"1min":246.16200000000001,"5min":246.16200000000001,"15min":246.16200000000001},"last":50.445999999999998},{"interface":"front","average":{"1min":21.619,"5min":20.536000000000001,"15min":20.823},"min":{"1min":8.0589999999999993,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":173.136,"5min":173.136,"15min":173.136},"last":10.507999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542604,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64220,"kb_used_data":11028,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307620,"statfs":{"total":96636764160,"available":96571002880,"internally_reserved":0,"allocated":11292672,"data_stored":6716024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":6.3659999999999997},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":8.0619999999999994}]},{"osd":1,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.869,"5min":15.02,"15min":851.14999999999998},"min":{"1min":7.2220000000000004,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":14.837999999999999,"5min":216.709,"15min":42105.105000000003},"last":18.609000000000002},{"interface":"front","average":{"1min":13.176,"5min":19.975999999999999,"15min":856.74199999999996},"min":{"1min":7.9480000000000004,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":17.890000000000001,"5min":217.929,"15min":42107.510000000002},"last":23.559000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":23.015999999999998},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":19.792999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.619,"5min":17.259,"15min":15.612},"min":{"1min":5.8739999999999997,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":16.402000000000001,"5min":149.886,"15min":172.952},"last":17.966000000000001},{"interface":"front","average":{"1min":15.183,"5min":24.073,"15min":18.233000000000001},"min":{"1min":10.442,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":25.030000000000001,"5min":231.691,"15min":231.691},"last":26.625}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19513.460999999999,"5min":8370.6499999999996,"15min":2802.931},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":8.5519999999999996},"max":{"1min":66230.460000000006,"5min":66734.365999999995,"15min":66734.365999999995},"last":20.399000000000001},{"interface":"front","average":{"1min":19500.227999999999,"5min":8373.2870000000003,"15min":2802.7069999999999},"min":{"1min":8.0199999999999996,"5min":7.282,"15min":7.1840000000000002},"max":{"1min":66215.267999999996,"5min":66719.994999999995,"15min":66719.994999999995},"last":27.327999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.661,"5min":21.093,"15min":2609.1860000000001},"min":{"1min":7.8140000000000001,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":24.298999999999999,"5min":247.161,"15min":74994.611999999994},"last":25.129000000000001},{"interface":"front","average":{"1min":13.272,"5min":20.702999999999999,"15min":2611.3249999999998},"min":{"1min":8.0670000000000002,"5min":8.0670000000000002,"15min":8.0670000000000002},"max":{"1min":21.920000000000002,"5min":241.91,"15min":74995.471000000005},"last":7.4080000000000004}]},{"osd":7,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":15.432,"5min":20.175999999999998,"15min":23.984999999999999},"min":{"1min":11.771000000000001,"5min":9.0150000000000006,"15min":9.0150000000000006},"max":{"1min":23.018000000000001,"5min":112.215,"15min":112.215},"last":19.236000000000001},{"interface":"front","average":{"1min":15.993,"5min":20.603000000000002,"15min":26.225999999999999},"min":{"1min":11.492000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":22.132999999999999,"5min":103.886,"15min":103.886},"last":22.161999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542605,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6361429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":9.8640000000000008,"5min":21.134,"15min":14.946},"min":{"1min":7.0199999999999996,"5min":4.8760000000000003,"15min":4.5640000000000001},"max":{"1min":16.805,"5min":302.62700000000001,"15min":302.62700000000001},"last":11.864000000000001},{"interface":"front","average":{"1min":13.249000000000001,"5min":24.818000000000001,"15min":18.748000000000001},"min":{"1min":8.6639999999999997,"5min":5.9379999999999997,"15min":5.9379999999999997},"max":{"1min":18.440999999999999,"5min":289.18200000000002,"15min":289.18200000000002},"last":17.309999999999999}]},{"osd":1,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":8.0830000000000002,"5min":26.745999999999999,"15min":1846.819},"min":{"1min":5.8609999999999998,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":12.907,"5min":682.15599999999995,"15min":57448.739999999998},"last":5.5780000000000003},{"interface":"front","average":{"1min":12.319000000000001,"5min":33.448,"15min":1853.625},"min":{"1min":8.2409999999999997,"5min":7.4470000000000001,"15min":7.4169999999999998},"max":{"1min":16.358000000000001,"5min":685.01300000000003,"15min":57479.919999999998},"last":9.2010000000000005}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":6.5359999999999996},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":11.598000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.497999999999999,"5min":28.012,"15min":18.911000000000001},"min":{"1min":8.6999999999999993,"5min":6.048,"15min":6.048},"max":{"1min":17.545000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":7.5499999999999998},{"interface":"front","average":{"1min":11.528,"5min":26.763000000000002,"15min":18.524000000000001},"min":{"1min":8.1560000000000006,"5min":6.4699999999999998,"15min":6.4050000000000002},"max":{"1min":18.917999999999999,"5min":609.31899999999996,"15min":609.31899999999996},"last":13.407}]},{"osd":5,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":14.292999999999999,"5min":40.664999999999999,"15min":26.305},"min":{"1min":8.8930000000000007,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":35.802,"5min":607.48199999999997,"15min":607.48199999999997},"last":12.811},{"interface":"front","average":{"1min":13.429,"5min":27.794,"15min":21.152999999999999},"min":{"1min":8.0269999999999992,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":37.572000000000003,"5min":287.95100000000002,"15min":592.32100000000003},"last":14.058}]},{"osd":6,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.945,"5min":26.32,"15min":2144.3380000000002},"min":{"1min":7.0549999999999997,"5min":7.0549999999999997,"15min":6.9480000000000004},"max":{"1min":16.335999999999999,"5min":688.24300000000005,"15min":75975.607000000004},"last":8.4900000000000002},{"interface":"front","average":{"1min":13.26,"5min":27.960999999999999,"15min":2143.8119999999999},"min":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.018000000000001,"5min":610.89800000000002,"15min":75971.296000000002},"last":14.66}]},{"osd":7,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":142.12799999999999,"15min":67.069000000000003},"min":{"1min":12.108000000000001,"5min":10.411,"15min":10.411},"max":{"1min":17.981000000000002,"5min":611.59799999999996,"15min":611.59799999999996},"last":15.207000000000001},{"interface":"front","average":{"1min":13.507999999999999,"5min":140.16900000000001,"15min":57.863},"min":{"1min":8.8960000000000008,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":18.225000000000001,"5min":629.07500000000005,"15min":629.07500000000005},"last":15.843999999999999}]}]},{"osd":3,"up_from":736,"seq":3161095930305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103948,"kb_used_data":10628,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267892,"statfs":{"total":96636764160,"available":96530321408,"internally_reserved":0,"allocated":10883072,"data_stored":6333298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.074999999999999,"5min":13.201000000000001,"15min":14.895},"min":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":101.90600000000001,"5min":136.09999999999999,"15min":159.828},"last":41.670000000000002},{"interface":"front","average":{"1min":12.532,"5min":13.233000000000001,"15min":14.949},"min":{"1min":6.0149999999999997,"5min":4.4000000000000004,"15min":4.4000000000000004},"max":{"1min":43.442999999999998,"5min":153.459,"15min":153.459},"last":40.825000000000003}]},{"osd":1,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":8.8539999999999992,"5min":13.349,"15min":386.524},"min":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":13.83,"5min":82.028000000000006,"15min":20852.026000000002},"last":152.60900000000001},{"interface":"front","average":{"1min":12.772,"5min":15.637,"15min":382.35199999999998},"min":{"1min":9.5039999999999996,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":27.445,"5min":141.65000000000001,"15min":20733.266},"last":61.618000000000002}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":192.08600000000001},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":179.96700000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":13.74,"5min":21.388999999999999,"15min":22.102},"min":{"1min":8.6859999999999999,"5min":7.7439999999999998,"15min":7.0129999999999999},"max":{"1min":31.504000000000001,"5min":194.845,"15min":201.905},"last":202.30099999999999},{"interface":"front","average":{"1min":14.554,"5min":21.510000000000002,"15min":22.814},"min":{"1min":8.1989999999999998,"5min":8.1199999999999992,"15min":6.944},"max":{"1min":33.781999999999996,"5min":196.65600000000001,"15min":210.239},"last":179.37299999999999}]},{"osd":5,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":18.071999999999999,"5min":21.088000000000001,"15min":24.696000000000002},"min":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":86.810000000000002,"5min":177.97999999999999,"15min":290.07100000000003},"last":180.82499999999999},{"interface":"front","average":{"1min":17.045000000000002,"5min":17.774999999999999,"15min":20.707000000000001},"min":{"1min":9.657,"5min":7.6619999999999999,"15min":6.8220000000000001},"max":{"1min":72.721999999999994,"5min":143.268,"15min":268.577},"last":117.535}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":17.827999999999999,"5min":18.077999999999999,"15min":1643.2470000000001},"min":{"1min":11.323,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":36.439999999999998,"5min":94.352000000000004,"15min":76086.231},"last":98.412999999999997},{"interface":"front","average":{"1min":15.978999999999999,"5min":20.367000000000001,"15min":1652.1669999999999},"min":{"1min":9.4280000000000008,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":56.807000000000002,"5min":146.048,"15min":76087.114000000001},"last":62.909999999999997}]},{"osd":7,"last update":"Thu Nov 2 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":16.010000000000002,"5min":19.359000000000002,"15min":23.548999999999999},"min":{"1min":10.382,"5min":8.8870000000000005,"15min":8.8870000000000005},"max":{"1min":40.081000000000003,"5min":152.32599999999999,"15min":152.32599999999999},"last":178.74299999999999},{"interface":"front","average":{"1min":16.681999999999999,"5min":19.952000000000002,"15min":24.228000000000002},"min":{"1min":9.5419999999999998,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":39.018000000000001,"5min":140.87299999999999,"15min":140.87299999999999},"last":118.634}]}]},{"osd":7,"up_from":875,"seq":3758096384429,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63840,"kb_used_data":10648,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94308000,"statfs":{"total":96636764160,"available":96571392000,"internally_reserved":0,"allocated":10903552,"data_stored":6325632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":10.130000000000001,"5min":13.368,"15min":15.576000000000001},"min":{"1min":6.1070000000000002,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":15.327,"5min":84.426000000000002,"15min":84.426000000000002},"last":5.4749999999999996},{"interface":"front","average":{"1min":8.7460000000000004,"5min":13.207000000000001,"15min":18.576000000000001},"min":{"1min":5.8650000000000002,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":14.715999999999999,"5min":97.984999999999999,"15min":97.984999999999999},"last":11.411}]},{"osd":1,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":12.630000000000001,"5min":16.503,"15min":22.439},"min":{"1min":7.5309999999999997,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":16.297000000000001,"5min":178.56,"15min":178.56},"last":6.2439999999999998},{"interface":"front","average":{"1min":9.2569999999999997,"5min":12.856999999999999,"15min":15.263},"min":{"1min":6.6589999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":11.67,"5min":66.081999999999994,"15min":66.081999999999994},"last":18.946999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":19.594999999999999},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":13.263}]},{"osd":3,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":14.497999999999999,"5min":20.501000000000001,"15min":19.623000000000001},"min":{"1min":8.9700000000000006,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":25.792000000000002,"5min":253.71000000000001,"15min":253.71000000000001},"last":7.6689999999999996},{"interface":"front","average":{"1min":12.202,"5min":16.843,"15min":18.457999999999998},"min":{"1min":7.9500000000000002,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":17.898,"5min":101.316,"15min":101.316},"last":20.265000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:44 2023","interfaces":[{"interface":"back","average":{"1min":15907.763999999999,"5min":9828.0190000000002,"15min":3302.183},"min":{"1min":8.968,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":65773.679000000004,"5min":66277.987999999998,"15min":66277.987999999998},"last":12.699999999999999},{"interface":"front","average":{"1min":15924.536,"5min":9833.3089999999993,"15min":3301.9609999999998},"min":{"1min":11.398999999999999,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":65797.989000000001,"5min":66301.380000000005,"15min":66301.380000000005},"last":9.0220000000000002}]},{"osd":5,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":13.279,"5min":21.350999999999999,"15min":22.777999999999999},"min":{"1min":8.9740000000000002,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":17.827999999999999,"5min":263.65699999999998,"15min":263.65699999999998},"last":20.821000000000002},{"interface":"front","average":{"1min":14.351000000000001,"5min":22.829999999999998,"15min":24.738},"min":{"1min":9.6669999999999998,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":19.327999999999999,"5min":183.96899999999999,"15min":183.96899999999999},"last":14.183}]},{"osd":6,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":18.141999999999999,"5min":24.789000000000001,"15min":26.550999999999998},"min":{"1min":10.255000000000001,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":72.542000000000002,"5min":285.56999999999999,"15min":285.56999999999999},"last":16.547000000000001},{"interface":"front","average":{"1min":17.75,"5min":22.82,"15min":23.931999999999999},"min":{"1min":9.7289999999999992,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":28.934000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":17.911000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:13:24.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-02T16:13:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.803+0000 7f99a9181640 10 monclient: get_monmap_and_config 2023-11-02T16:13:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.803+0000 7f99a9181640 10 monclient: build_initial_monmap 2023-11-02T16:13:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.803+0000 7f99a9181640 10 monclient: monmap: 2023-11-02T16:13:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:24.804548+0000 2023-11-02T16:13:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:24.804548+0000 2023-11-02T16:13:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:24.807 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.807 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:24.807 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:24.807 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:24.807 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:24.808 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:24.808 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.811 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 Processor -- start 2023-11-02T16:13:24.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 -- start start 2023-11-02T16:13:24.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient: init 2023-11-02T16:13:24.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:24.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient: _add_conns ranks=[4,2,1,0,3] 2023-11-02T16:13:24.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99a40b4170 0x7f99a4141800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-c con 0x7f99a40b4170 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:24.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-b con 0x7f99a40a8df0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:24.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f99a4142280 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a40aa3d0 con 0x7f99a4142280 2023-11-02T16:13:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a413f420 con 0x7f99a40b4170 2023-11-02T16:13:24.817 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40b4550 con 0x7f99a40a8df0 2023-11-02T16:13:24.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:24.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.804+0000 7f99a9181640 10 monclient(hunting): authenticate will time out at 3732.146616s 2023-11-02T16:13:24.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f99a2d9d640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99a40b4170 0x7f99a4141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46362/0 (socket says 172.21.15.174:46362) 2023-11-02T16:13:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 1 -- 172.21.15.174:0/973346417 learned_addr learned my addr 172.21.15.174:0/973346417 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 10 monclient(hunting): get_auth_request con 0x7f99a40a8df0 auth_method 0 2023-11-02T16:13:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.805+0000 7f999bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): get_auth_request con 0x7f99a40b4170 auth_method 0 2023-11-02T16:13:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.806+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 10 monclient(hunting): handle_auth_done global_id 16901 payload 274 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 1 -- 172.21.15.174:0/973346417 >> v2:172.21.15.175:3300/0 conn(0x7f99a40b4170 msgr2=0x7f99a4141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:24.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 1 --2- 172.21.15.174:0/973346417 >> v2:172.21.15.175:3300/0 conn(0x7f99a40b4170 0x7f99a4141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 1 -- 172.21.15.174:0/973346417 >> v1:172.21.15.175:6789/0 conn(0x7f99a4142280 legacy=0x7f99a40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 10 monclient: _finish_hunting 0 2023-11-02T16:13:24.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 1 monclient: found mon.noname-b 2023-11-02T16:13:24.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:24.824 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:24.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.808+0000 7f999bfff640 1 -- 172.21.15.174:0/973346417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40a92d0 con 0x7f99a40a8df0 2023-11-02T16:13:24.825 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.810+0000 7f999bfff640 1 --2- 172.21.15.174:0/973346417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f998c009870 tx=0x7f998c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e5da7b1341d682 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:24.830 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 1 -- 172.21.15.174:0/973346417 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c031640 con 0x7f99a40a8df0 2023-11-02T16:13:24.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:24.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: dump: 2023-11-02T16:13:24.831 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.831 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: _finish_auth 0 2023-11-02T16:13:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: _check_auth_tickets 2023-11-02T16:13:24.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:24.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 1 -- 172.21.15.174:0/973346417 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c0317a0 con 0x7f99a40a8df0 2023-11-02T16:13:24.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a9181640 5 monclient: authenticate success, global_id 16901 2023-11-02T16:13:24.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a9181640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:24.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:24.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 1 -- 172.21.15.174:0/973346417 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c031b30 con 0x7f99a40a8df0 2023-11-02T16:13:24.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a9181640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:24.835 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a9181640 4 set_mon_vals no callback set 2023-11-02T16:13:24.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:24.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.813+0000 7f99a259c640 10 monclient: dump: 2023-11-02T16:13:24.836 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.836 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.837 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.837 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.837 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.837 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.837 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.838 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.838 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.838 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 10 monclient: shutdown 2023-11-02T16:13:24.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 -- 172.21.15.174:0/973346417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 msgr2=0x7f99a4141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 --2- 172.21.15.174:0/973346417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f998c009870 tx=0x7f998c02b8f0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 -- 172.21.15.174:0/973346417 shutdown_connections 2023-11-02T16:13:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 --2- 172.21.15.174:0/973346417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40a8df0 0x7f99a4141d40 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 --2- 172.21.15.174:0/973346417 >> v2:172.21.15.175:3300/0 conn(0x7f99a40b4170 0x7f99a4141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.840 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 -- 172.21.15.174:0/973346417 >> 172.21.15.174:0/973346417 conn(0x7f99a4065d80 msgr2=0x7f99a4066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:24.840 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 -- 172.21.15.174:0/973346417 shutdown_connections 2023-11-02T16:13:24.840 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.814+0000 7f99a9181640 1 -- 172.21.15.174:0/973346417 wait complete. 2023-11-02T16:13:24.840 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 10 monclient: build_initial_monmap 2023-11-02T16:13:24.841 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 10 monclient: monmap: 2023-11-02T16:13:24.841 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:24.841 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:24.841 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:24.841 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:24.842 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 librados: starting msgr at 2023-11-02T16:13:24.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 librados: starting objecter 2023-11-02T16:13:24.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 Processor -- start 2023-11-02T16:13:24.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 -- start start 2023-11-02T16:13:24.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 librados: setting wanted keys 2023-11-02T16:13:24.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 1 librados: calling monclient init 2023-11-02T16:13:24.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.815+0000 7f99a9181640 10 monclient: init 2023-11-02T16:13:24.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:24.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T16:13:24.844 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99a40a8df0 0x7f99a40ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-a con 0x7f99a40a8df0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:24.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40b4170 0x7f99a41d6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-c con 0x7f99a40b4170 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:24.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:24.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): picked mon.noname-b con 0x7f99a4142280 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:24.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40ac430 con 0x7f99a40a8df0 2023-11-02T16:13:24.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40ad6e0 con 0x7f99a4142280 2023-11-02T16:13:24.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:24.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a41453e0 con 0x7f99a40b4170 2023-11-02T16:13:24.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:24.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a9181640 10 monclient(hunting): authenticate will time out at 3732.158013s 2023-11-02T16:13:24.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a359e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.816+0000 7f99a2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99a40a8df0 0x7f99a40ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f999bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40b4170 0x7f99a41d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:24.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:39512/0 (socket says 172.21.15.174:39512) 2023-11-02T16:13:24.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 1 -- 172.21.15.174:0/3630506513 learned_addr learned my addr 172.21.15.174:0/3630506513 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:24.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): get_auth_request con 0x7f99a4142280 auth_method 0 2023-11-02T16:13:24.848 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a2d9d640 10 monclient(hunting): get_auth_request con 0x7f99a40a8df0 auth_method 0 2023-11-02T16:13:24.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a2d9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a2d9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f999bfff640 10 monclient(hunting): get_auth_request con 0x7f99a40b4170 auth_method 0 2023-11-02T16:13:24.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f999bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:24.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f999bfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:24.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f999bfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.817+0000 7f99a359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f999bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:24.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:24.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.818+0000 7f99a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:24.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 10 monclient(hunting): handle_auth_done global_id 21502 payload 824 2023-11-02T16:13:24.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 -- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40b4170 msgr2=0x7f99a41d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40b4170 0x7f99a41d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 -- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99a40a8df0 msgr2=0x7f99a40ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:24.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99a40a8df0 0x7f99a40ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:24.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 10 monclient: _finish_hunting 0 2023-11-02T16:13:24.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 monclient: found mon.noname-b 2023-11-02T16:13:24.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:24.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.820+0000 7f99a359e640 1 -- 172.21.15.174:0/3630506513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c0094d0 con 0x7f99a4142280 2023-11-02T16:13:24.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.822+0000 7f99a359e640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 secure :-1 s=READY pgs=5258 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c760 tx=0x7f999c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:24.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.822+0000 7f999b7fe640 10 client.?.objecter ms_handle_connect 0x7f99a4142280 2023-11-02T16:13:24.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.822+0000 7f999b7fe640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:24.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c006d50 con 0x7f99a4142280 2023-11-02T16:13:24.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:24.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 10 monclient: dump: 2023-11-02T16:13:24.856 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.856 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.856 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.856 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.856 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.857 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.857 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.857 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.857 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.857 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f999b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 5 monclient: authenticate success, global_id 21502 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 10 monclient: _renew_subs 2023-11-02T16:13:24.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40abe70 con 0x7f99a4142280 2023-11-02T16:13:24.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 10 client.21502.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:24.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 10 monclient: _renew_subs 2023-11-02T16:13:24.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41d73c0 con 0x7f99a4142280 2023-11-02T16:13:24.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.824+0000 7f99a9181640 1 librados: init done 2023-11-02T16:13:24.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.825+0000 7f99a9181640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:24.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.825+0000 7f99a9181640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:24.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.825+0000 7f99a9181640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:24.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.825+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a40ac570 con 0x7f99a4142280 2023-11-02T16:13:24.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c0016f0 con 0x7f99a4142280 2023-11-02T16:13:24.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:24.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c003b90 con 0x7f99a4142280 2023-11-02T16:13:24.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:24.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:24.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f999b7fe640 10 monclient: dump: 2023-11-02T16:13:24.862 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:24.862 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:24.862 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.863 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:24.863 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:24.863 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:24.863 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:24.863 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:24.864 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:24.864 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:24.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:24.827+0000 7f99a1d9b640 4 set_mon_vals no callback set 2023-11-02T16:13:25.072 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.070+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f999c003cf0 con 0x7f99a4142280 2023-11-02T16:13:25.072 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.071+0000 7f999b7fe640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 0x7f998406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:25.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.071+0000 7f99a2d9d640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 0x7f998406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:25.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.071+0000 7f99a2d9d640 10 monclient: get_auth_request con 0x7f998406bdc0 auth_method 0 2023-11-02T16:13:25.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.071+0000 7f99a2d9d640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 0x7f998406e280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9994009b40 tx=0x7f9994009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:25.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.071+0000 7f999b7fe640 10 client.21502.objecter ms_handle_connect 0x7f998406bdc0 2023-11-02T16:13:25.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.074+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f999c0582a0 con 0x7f99a4142280 2023-11-02T16:13:25.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.074+0000 7f999b7fe640 10 client.21502.objecter ms_dispatch 0x7f99a4000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:25.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.074+0000 7f999b7fe640 3 client.21502.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:25.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.074+0000 7f999b7fe640 3 client.21502.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:25.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.074+0000 7f999b7fe640 20 client.21502.objecter dump_active .. 0 homeless 2023-11-02T16:13:25.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.083+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f999c0545e0 con 0x7f99a4142280 2023-11-02T16:13:25.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.083+0000 7f999b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:25.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.083+0000 7f999b7fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:25.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.253+0000 7f99a9181640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-11-02T16:13:25.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.253+0000 7f99a9181640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T16:13:25.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.253+0000 7f99a9181640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:25.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.253+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a40ac570 con 0x7f99a4142280 2023-11-02T16:13:25.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.289+0000 7f999b7fe640 1 -- 172.21.15.174:0/3630506513 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v1030) v1 ==== 74+0+17349 (secure 0 0 0) 0x7f99a40ac570 con 0x7f99a4142280 2023-11-02T16:13:25.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.289+0000 7f999b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-11-02T16:13:25.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.289+0000 7f999b7fe640 10 monclient: _finish_command 2 = system:0 2023-11-02T16:13:25.292 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:25.292 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1030,"fsid":"71bb545f-71ad-46c8-9016-794bb3baf3fa","created":"2023-11-02T15:33:05.250770+0000","modified":"2023-11-02T16:13:24.249672+0000","last_up_change":"2023-11-02T16:13:15.327523+0000","last_in_change":"2023-11-02T15:33:10.458261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":106,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":259,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-02T15:33:53.130231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-02T15:33:59.744250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"ceph_test_rados_api_asio","create_time":"2023-11-02T15:35:06.951222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8cc58e83-ddd7-4452-881d-72078e5d9891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":1023,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":86149},{"type":"v1","addr":"172.21.15.174:6803","nonce":86149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":86149},{"type":"v1","addr":"172.21.15.174:6805","nonce":86149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":86149},{"type":"v1","addr":"172.21.15.174:6809","nonce":86149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":86149},{"type":"v1","addr":"172.21.15.174:6807","nonce":86149}]},"public_addr":"172.21.15.174:6803/86149","cluster_addr":"172.21.15.174:6805/86149","heartbeat_back_addr":"172.21.15.174:6809/86149","heartbeat_front_addr":"172.21.15.174:6807/86149","state":["exists","up"]},{"osd":1,"uuid":"8be55432-eb0e-4795-8de9-3831cc022e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":1023,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":86146},{"type":"v1","addr":"172.21.15.174:6821","nonce":86146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":86146},{"type":"v1","addr":"172.21.15.174:6827","nonce":86146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":86146},{"type":"v1","addr":"172.21.15.174:6833","nonce":86146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":86146},{"type":"v1","addr":"172.21.15.174:6831","nonce":86146}]},"public_addr":"172.21.15.174:6821/86146","cluster_addr":"172.21.15.174:6827/86146","heartbeat_back_addr":"172.21.15.174:6833/86146","heartbeat_front_addr":"172.21.15.174:6831/86146","state":["exists","up"]},{"osd":2,"uuid":"396e4073-1411-4e20-b491-1f63a71809a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":16,"last_clean_end":1021,"up_from":1023,"up_thru":1023,"down_at":1019,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":86153},{"type":"v1","addr":"172.21.15.174:6811","nonce":86153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2086153},{"type":"v1","addr":"172.21.15.174:6823","nonce":2086153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":86153},{"type":"v1","addr":"172.21.15.174:6817","nonce":86153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":86153},{"type":"v1","addr":"172.21.15.174:6815","nonce":86153}]},"public_addr":"172.21.15.174:6811/86153","cluster_addr":"172.21.15.174:6823/2086153","heartbeat_back_addr":"172.21.15.174:6817/86153","heartbeat_front_addr":"172.21.15.174:6815/86153","state":["exists","up"]},{"osd":3,"uuid":"751b6bb7-19d9-47ab-a641-5559f3e6d131","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":734,"up_from":736,"up_thru":1023,"down_at":729,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":86148},{"type":"v1","addr":"172.21.15.174:6820","nonce":86148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1086148},{"type":"v1","addr":"172.21.15.174:6813","nonce":1086148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":86148},{"type":"v1","addr":"172.21.15.174:6829","nonce":86148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":86148},{"type":"v1","addr":"172.21.15.174:6826","nonce":86148}]},"public_addr":"172.21.15.174:6820/86148","cluster_addr":"172.21.15.174:6813/1086148","heartbeat_back_addr":"172.21.15.174:6829/86148","heartbeat_front_addr":"172.21.15.174:6826/86148","state":["exists","up"]},{"osd":4,"uuid":"a08d368b-40c5-4e74-b125-dcf45d0e9fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":1023,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":86415},{"type":"v1","addr":"172.21.15.175:6817","nonce":86415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":86415},{"type":"v1","addr":"172.21.15.175:6819","nonce":86415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":86415},{"type":"v1","addr":"172.21.15.175:6823","nonce":86415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":86415},{"type":"v1","addr":"172.21.15.175:6821","nonce":86415}]},"public_addr":"172.21.15.175:6817/86415","cluster_addr":"172.21.15.175:6819/86415","heartbeat_back_addr":"172.21.15.175:6823/86415","heartbeat_front_addr":"172.21.15.175:6821/86415","state":["exists","up"]},{"osd":5,"uuid":"27014a04-b88c-4215-ba2f-9d2865093de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":1023,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":86416},{"type":"v1","addr":"172.21.15.175:6809","nonce":86416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":86416},{"type":"v1","addr":"172.21.15.175:6811","nonce":86416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":86416},{"type":"v1","addr":"172.21.15.175:6815","nonce":86416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":86416},{"type":"v1","addr":"172.21.15.175:6813","nonce":86416}]},"public_addr":"172.21.15.175:6809/86416","cluster_addr":"172.21.15.175:6811/86416","heartbeat_back_addr":"172.21.15.175:6815/86416","heartbeat_front_addr":"172.21.15.175:6813/86416","state":["exists","up"]},{"osd":6,"uuid":"1edde1fd-2330-42d8-a7bb-015cebd9d38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":385,"up_from":387,"up_thru":1023,"down_at":333,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":86417},{"type":"v1","addr":"172.21.15.175:6825","nonce":86417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1086417},{"type":"v1","addr":"172.21.15.175:6833","nonce":1086417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":86417},{"type":"v1","addr":"172.21.15.175:6831","nonce":86417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":86417},{"type":"v1","addr":"172.21.15.175:6829","nonce":86417}]},"public_addr":"172.21.15.175:6825/86417","cluster_addr":"172.21.15.175:6833/1086417","heartbeat_back_addr":"172.21.15.175:6831/86417","heartbeat_front_addr":"172.21.15.175:6829/86417","state":["exists","up"]},{"osd":7,"uuid":"a0a6eebf-5d56-4899-83ec-b257dd65d940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":874,"up_from":875,"up_thru":1023,"down_at":871,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":86418},{"type":"v1","addr":"172.21.15.175:6801","nonce":86418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2086418},{"type":"v1","addr":"172.21.15.175:6803","nonce":2086418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":86418},{"type":"v1","addr":"172.21.15.175:6807","nonce":86418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":86418},{"type":"v1","addr":"172.21.15.175:6805","nonce":86418}]},"public_addr":"172.21.15.175:6801/86418","cluster_addr":"172.21.15.175:6803/2086418","heartbeat_back_addr":"172.21.15.175:6807/86418","heartbeat_front_addr":"172.21.15.175:6805/86418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.105568+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:49.744420+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-11-02T16:13:11.395316+0000","laggy_probability":0.50999999046325684,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:50.135477+0000","dead_epoch":1020},{"osd":3,"down_stamp":"2023-11-02T15:59:08.055146+0000","laggy_probability":0.30000001192092896,"laggy_interval":2,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.453210+0000","dead_epoch":733},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.626045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.898023+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-11-02T15:42:57.860686+0000","laggy_probability":0.30000001192092896,"laggy_interval":16,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:48.470083+0000","dead_epoch":343},{"osd":7,"down_stamp":"2023-11-02T16:07:21.435277+0000","laggy_probability":0.50999999046325684,"laggy_interval":2,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-02T15:33:47.661882+0000","dead_epoch":873}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":11,"snaps":[{"begin":7,"length":1},{"begin":9,"length":2}]},{"pool":54,"snaps":[{"begin":11,"length":2}]},{"pool":83,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":88,"snaps":[{"begin":2,"length":1}]},{"pool":95,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2}]},{"pool":101,"snaps":[{"begin":4,"length":4}]},{"pool":108,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":118,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":179,"snaps":[{"begin":12,"length":1}]},{"pool":189,"snaps":[{"begin":14,"length":1}]},{"pool":193,"snaps":[{"begin":17,"length":1}]},{"pool":194,"snaps":[{"begin":19,"length":2}]},{"pool":197,"snaps":[{"begin":24,"length":1}]},{"pool":240,"snaps":[{"begin":2,"length":1}]},{"pool":241,"snaps":[{"begin":4,"length":1}]},{"pool":244,"snaps":[{"begin":6,"length":1}]},{"pool":248,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-02T16:13:25.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 10 librados: watch_flush enter 2023-11-02T16:13:25.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 10 librados: watch_flush exit 2023-11-02T16:13:25.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 20 client.21502.objecter shutdown clearing up homeless session... 2023-11-02T16:13:25.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 10 client.21502.objecter successfully canceled tick 2023-11-02T16:13:25.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 msgr2=0x7f998406e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:25.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 0x7f998406e280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9994009b40 tx=0x7f9994009190 comp rx=0 tx=0).stop 2023-11-02T16:13:25.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 10 monclient: shutdown 2023-11-02T16:13:25.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:25.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 msgr2=0x7f99a41d6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:25.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 secure :-1 s=READY pgs=5258 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c760 tx=0x7f999c00cc20 comp rx=0 tx=0).stop 2023-11-02T16:13:25.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 shutdown_connections 2023-11-02T16:13:25.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99a4142280 0x7f99a41d6ca0 unknown :-1 s=CLOSED pgs=5258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:25.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f998406bdc0 0x7f998406e280 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:25.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99a40b4170 0x7f99a41d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:25.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 --2- 172.21.15.174:0/3630506513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99a40a8df0 0x7f99a40ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:25.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 >> 172.21.15.174:0/3630506513 conn(0x7f99a4065d80 msgr2=0x7f99a40b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:25.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 shutdown_connections 2023-11-02T16:13:25.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 -- 172.21.15.174:0/3630506513 wait complete. 2023-11-02T16:13:25.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:25.291+0000 7f99a9181640 1 librados: shutdown 2023-11-02T16:13:26.305 INFO:tasks.ceph:Scrubbing osd.0 2023-11-02T16:13:26.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:26.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 10 monclient: get_monmap_and_config 2023-11-02T16:13:26.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 10 monclient: build_initial_monmap 2023-11-02T16:13:26.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 10 monclient: monmap: 2023-11-02T16:13:26.411 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:26.411 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.412 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:26.410073+0000 2023-11-02T16:13:26.412 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:26.410073+0000 2023-11-02T16:13:26.412 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:26.412 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.412 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:26.413 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:26.413 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:26.413 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:26.413 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:26.413 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 1 Processor -- start 2023-11-02T16:13:26.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 1 -- start start 2023-11-02T16:13:26.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.409+0000 7fd9745fd640 10 monclient: init 2023-11-02T16:13:26.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:26.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2023-11-02T16:13:26.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-b con 0x7fd96c0b3c10 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:26.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd96c13ede0 0x7fd96c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-c con 0x7fd96c13ede0 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:26.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd96c0b0a00 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:26.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0af970 con 0x7fd96c0b0a00 2023-11-02T16:13:26.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0b3ff0 con 0x7fd96c13ede0 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0afbc0 con 0x7fd96c0b3c10 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.410+0000 7fd9745fd640 10 monclient(hunting): authenticate will time out at 3733.752213s 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd971b98640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd96c13ede0 0x7fd96c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54580/0 (socket says 172.21.15.174:54580) 2023-11-02T16:13:26.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 1 -- 172.21.15.174:0/853318560 learned_addr learned my addr 172.21.15.174:0/853318560 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:26.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 10 monclient(hunting): get_auth_request con 0x7fd96c0b3c10 auth_method 0 2023-11-02T16:13:26.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd972399640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd971b98640 10 monclient(hunting): get_auth_request con 0x7fd96c13ede0 auth_method 0 2023-11-02T16:13:26.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd971b98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd971b98640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.411+0000 7fd971b98640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.412+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 10 monclient(hunting): handle_auth_done global_id 16907 payload 274 2023-11-02T16:13:26.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 1 -- 172.21.15.174:0/853318560 >> v2:172.21.15.175:3300/0 conn(0x7fd96c13ede0 msgr2=0x7fd96c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:26.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 1 --2- 172.21.15.174:0/853318560 >> v2:172.21.15.175:3300/0 conn(0x7fd96c13ede0 0x7fd96c0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 1 -- 172.21.15.174:0/853318560 >> v1:172.21.15.175:6789/0 conn(0x7fd96c0b0a00 legacy=0x7fd96c0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 10 monclient: _finish_hunting 0 2023-11-02T16:13:26.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 1 monclient: found mon.noname-b 2023-11-02T16:13:26.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:26.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.413+0000 7fd972399640 1 -- 172.21.15.174:0/853318560 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0afea0 con 0x7fd96c0b3c10 2023-11-02T16:13:26.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.415+0000 7fd972399640 1 --2- 172.21.15.174:0/853318560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009870 tx=0x7fd95c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4d649168d6374d server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.418+0000 7fd971397640 1 -- 172.21.15.174:0/853318560 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c031640 con 0x7fd96c0b3c10 2023-11-02T16:13:26.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:26.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: dump: 2023-11-02T16:13:26.431 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.431 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.431 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.432 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.432 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.432 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.432 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.432 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.433 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.433 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: _finish_auth 0 2023-11-02T16:13:26.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: _check_auth_tickets 2023-11-02T16:13:26.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:26.434 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 1 -- 172.21.15.174:0/853318560 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c0317a0 con 0x7fd96c0b3c10 2023-11-02T16:13:26.434 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 5 monclient: authenticate success, global_id 16907 2023-11-02T16:13:26.434 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:26.434 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:26.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 1 -- 172.21.15.174:0/853318560 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c031b30 con 0x7fd96c0b3c10 2023-11-02T16:13:26.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:26.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 4 set_mon_vals no callback set 2023-11-02T16:13:26.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd971397640 10 monclient: dump: 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.437 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.437 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.437 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.437 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.437 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 10 monclient: shutdown 2023-11-02T16:13:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 -- 172.21.15.174:0/853318560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 msgr2=0x7fd96c13e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 --2- 172.21.15.174:0/853318560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009870 tx=0x7fd95c02b8f0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 -- 172.21.15.174:0/853318560 shutdown_connections 2023-11-02T16:13:26.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 --2- 172.21.15.174:0/853318560 >> v2:172.21.15.175:3300/0 conn(0x7fd96c13ede0 0x7fd96c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 --2- 172.21.15.174:0/853318560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c0b3c10 0x7fd96c13e8a0 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 -- 172.21.15.174:0/853318560 >> 172.21.15.174:0/853318560 conn(0x7fd96c065960 msgr2=0x7fd96c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:26.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 -- 172.21.15.174:0/853318560 shutdown_connections 2023-11-02T16:13:26.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.419+0000 7fd9745fd640 1 -- 172.21.15.174:0/853318560 wait complete. 2023-11-02T16:13:26.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.420+0000 7fd9745fd640 10 monclient: build_initial_monmap 2023-11-02T16:13:26.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.420+0000 7fd9745fd640 10 monclient: monmap: 2023-11-02T16:13:26.440 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:26.440 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:26.440 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:26.441 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:26.441 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:26.441 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.441 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:26.441 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.420+0000 7fd9745fd640 1 librados: starting msgr at 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.420+0000 7fd9745fd640 1 librados: starting objecter 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.420+0000 7fd9745fd640 1 Processor -- start 2023-11-02T16:13:26.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 -- start start 2023-11-02T16:13:26.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 librados: setting wanted keys 2023-11-02T16:13:26.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 librados: calling monclient init 2023-11-02T16:13:26.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient: init 2023-11-02T16:13:26.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:26.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T16:13:26.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c0b0a00 0x7fd96c13aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-a con 0x7fd96c0b0a00 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:26.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c0b3c10 0x7fd96c13afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-b con 0x7fd96c0b3c10 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:26.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 0x7fd96c1d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): picked mon.noname-c con 0x7fd96c13ede0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:26.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1d6610 con 0x7fd96c0b0a00 2023-11-02T16:13:26.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1391f0 con 0x7fd96c0b3c10 2023-11-02T16:13:26.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c143070 con 0x7fd96c13ede0 2023-11-02T16:13:26.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:26.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.421+0000 7fd9745fd640 10 monclient(hunting): authenticate will time out at 3733.763230s 2023-11-02T16:13:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 0x7fd96c1d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972399640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c0b0a00 0x7fd96c13aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 0x7fd96c1d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54596/0 (socket says 172.21.15.174:54596) 2023-11-02T16:13:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 1 -- 172.21.15.174:0/423023842 learned_addr learned my addr 172.21.15.174:0/423023842 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): get_auth_request con 0x7fd96c13ede0 auth_method 0 2023-11-02T16:13:26.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd971b98640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c0b3c10 0x7fd96c13afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972399640 10 monclient(hunting): get_auth_request con 0x7fd96c0b0a00 auth_method 0 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972399640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972399640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972399640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd971b98640 10 monclient(hunting): get_auth_request con 0x7fd96c0b3c10 auth_method 0 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd971b98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd971b98640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd971b98640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.422+0000 7fd972b9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd972399640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.423+0000 7fd971b98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 10 monclient(hunting): handle_auth_done global_id 16910 payload 824 2023-11-02T16:13:26.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 -- 172.21.15.174:0/423023842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c0b3c10 msgr2=0x7fd96c13afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd96c0b3c10 0x7fd96c13afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 -- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c0b0a00 msgr2=0x7fd96c13aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c0b0a00 0x7fd96c13aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 10 monclient: _finish_hunting 0 2023-11-02T16:13:26.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 monclient: found mon.noname-c 2023-11-02T16:13:26.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:26.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.425+0000 7fd972b9a640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0094d0 con 0x7fd96c13ede0 2023-11-02T16:13:26.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.427+0000 7fd972b9a640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 0x7fd96c1d6a60 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c800 tx=0x7fd96400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.427+0000 7fd962ffd640 10 client.?.objecter ms_handle_connect 0x7fd96c13ede0 2023-11-02T16:13:26.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.427+0000 7fd962ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:26.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964003b70 con 0x7fd96c13ede0 2023-11-02T16:13:26.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:26.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: dump: 2023-11-02T16:13:26.457 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.457 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.457 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.458 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.458 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.458 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.462 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.462 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.462 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.463 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:26.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:26.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:26.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964003cf0 con 0x7fd96c13ede0 2023-11-02T16:13:26.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:26.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400aa90 con 0x7fd96c13ede0 2023-11-02T16:13:26.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:26.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd962ffd640 10 monclient: dump: 2023-11-02T16:13:26.465 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.465 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.465 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.465 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.465 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.466 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.466 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.466 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.466 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.466 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 5 monclient: authenticate success, global_id 16910 2023-11-02T16:13:26.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 10 monclient: _renew_subs 2023-11-02T16:13:26.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c139d50 con 0x7fd96c13ede0 2023-11-02T16:13:26.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd970b96640 4 set_mon_vals no callback set 2023-11-02T16:13:26.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 10 client.16910.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:26.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 10 monclient: _renew_subs 2023-11-02T16:13:26.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c1d6fa0 con 0x7fd96c13ede0 2023-11-02T16:13:26.468 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.431+0000 7fd9745fd640 1 librados: init done 2023-11-02T16:13:26.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 10 client.16910.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:26.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 15 client.16910.objecter _session_command_op_assign -1 1 2023-11-02T16:13:26.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 20 client.16910.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:26.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 10 client.16910.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:26.469 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.432+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd96c13f220 con 0x7fd96c13ede0 2023-11-02T16:13:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.450+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd96400abf0 con 0x7fd96c13ede0 2023-11-02T16:13:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.450+0000 7fd962ffd640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd95006be10 0x7fd95006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.451+0000 7fd972399640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd95006be10 0x7fd95006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.451+0000 7fd972399640 10 monclient: get_auth_request con 0x7fd95006be10 auth_method 0 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.451+0000 7fd972399640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd95006be10 0x7fd95006e2d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd95c036000 tx=0x7fd95c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.451+0000 7fd962ffd640 10 client.16910.objecter ms_handle_connect 0x7fd95006be10 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fd964058340 con 0x7fd96c13ede0 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 10 client.16910.objecter ms_dispatch 0x7fd96c000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 3 client.16910.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:26.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 3 client.16910.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:26.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 10 client.16910.objecter checking command 1 2023-11-02T16:13:26.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7fd950073780 0x7fd950075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 20 client.16910.objecter _get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 15 client.16910.objecter _session_command_op_remove -1 1 2023-11-02T16:13:26.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 20 client.16910.objecter _get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 20 client.16910.objecter get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 15 client.16910.objecter _session_command_op_assign 0 1 2023-11-02T16:13:26.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.454+0000 7fd962ffd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 4 2023-11-02T16:13:26.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd962ffd640 10 client.16910.objecter _send_command 1 2023-11-02T16:13:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9500762f0 con 0x7fd950073780 2023-11-02T16:13:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd962ffd640 20 client.16910.objecter dump_active .. 0 homeless 2023-11-02T16:13:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd96400aea0 con 0x7fd96c13ede0 2023-11-02T16:13:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd962ffd640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:26.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd971b98640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7fd950073780 0x7fd950075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.455+0000 7fd971b98640 10 monclient: get_auth_request con 0x7fd950073780 auth_method 0 2023-11-02T16:13:26.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.458+0000 7fd971b98640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7fd950073780 0x7fd950075c20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.458+0000 7fd962ffd640 10 client.16910.objecter ms_handle_connect 0x7fd950073780 2023-11-02T16:13:26.481 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.480+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== osd.0 v2:172.21.15.174:6802/86149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd9500762f0 con 0x7fd950073780 2023-11-02T16:13:26.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.480+0000 7fd962ffd640 10 client.16910.objecter ms_dispatch 0x7fd96c000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:26.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.480+0000 7fd962ffd640 10 client.16910.objecter _finish_command 1 = system:0 2023-11-02T16:13:26.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.480+0000 7fd962ffd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 4 2023-11-02T16:13:26.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.480+0000 7fd962ffd640 15 client.16910.objecter _session_command_op_remove 0 1 2023-11-02T16:13:26.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 10 client.16910.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:26.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 15 client.16910.objecter _session_command_op_assign -1 2 2023-11-02T16:13:26.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 20 client.16910.objecter _get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 20 client.16910.objecter _get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 15 client.16910.objecter _session_command_op_remove -1 2 2023-11-02T16:13:26.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 20 client.16910.objecter get_session s=0x7fd9500730b0 osd=0 3 2023-11-02T16:13:26.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 15 client.16910.objecter _session_command_op_assign 0 2 2023-11-02T16:13:26.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 4 2023-11-02T16:13:26.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 10 client.16910.objecter _send_command 2 2023-11-02T16:13:26.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.492+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 --> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd96c0ae380 con 0x7fd950073780 2023-11-02T16:13:26.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd962ffd640 1 -- 172.21.15.174:0/423023842 <== osd.0 v2:172.21.15.174:6802/86149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd968009bd0 con 0x7fd950073780 2023-11-02T16:13:26.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd962ffd640 10 client.16910.objecter ms_dispatch 0x7fd96c000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:26.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd962ffd640 10 client.16910.objecter _finish_command 2 = system:0 2023-11-02T16:13:26.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd962ffd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 4 2023-11-02T16:13:26.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd962ffd640 15 client.16910.objecter _session_command_op_remove 0 2 2023-11-02T16:13:26.509 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:26.509 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:26.509 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:26.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 10 librados: watch_flush enter 2023-11-02T16:13:26.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 10 librados: watch_flush exit 2023-11-02T16:13:26.511 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 10 client.16910.objecter close_session for osd.0 2023-11-02T16:13:26.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7fd950073780 msgr2=0x7fd950075c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6802/86149,v1:172.21.15.174:6803/86149] conn(0x7fd950073780 0x7fd950075c20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 20 client.16910.objecter put_session s=0x7fd9500730b0 osd=0 1 2023-11-02T16:13:26.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 20 client.16910.objecter shutdown clearing up homeless session... 2023-11-02T16:13:26.512 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 10 client.16910.objecter successfully canceled tick 2023-11-02T16:13:26.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd95006be10 msgr2=0x7fd95006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd95006be10 0x7fd95006e2d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd95c036000 tx=0x7fd95c03c040 comp rx=0 tx=0).stop 2023-11-02T16:13:26.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 10 monclient: shutdown 2023-11-02T16:13:26.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:26.513 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 msgr2=0x7fd96c1d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 --2- 172.21.15.174:0/423023842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96c13ede0 0x7fd96c1d6a60 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c800 tx=0x7fd96400ccc0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd972b9a640 1 -- 172.21.15.174:0/423023842 reap_dead start 2023-11-02T16:13:26.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 shutdown_connections 2023-11-02T16:13:26.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 >> 172.21.15.174:0/423023842 conn(0x7fd96c065960 msgr2=0x7fd96c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:26.514 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 shutdown_connections 2023-11-02T16:13:26.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 -- 172.21.15.174:0/423023842 wait complete. 2023-11-02T16:13:26.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.506+0000 7fd9745fd640 1 librados: shutdown 2023-11-02T16:13:26.519 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-02T16:13:26.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.619+0000 7f92cc59f640 10 monclient: get_monmap_and_config 2023-11-02T16:13:26.621 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.619+0000 7f92cc59f640 10 monclient: build_initial_monmap 2023-11-02T16:13:26.622 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.619+0000 7f92cc59f640 10 monclient: monmap: 2023-11-02T16:13:26.622 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:26.622 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.622 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:26.620672+0000 2023-11-02T16:13:26.622 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:26.620672+0000 2023-11-02T16:13:26.623 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:26.623 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.623 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:26.623 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:26.623 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:26.624 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:26.624 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:26.624 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.626 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.620+0000 7f92cc59f640 1 Processor -- start 2023-11-02T16:13:26.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.620+0000 7f92cc59f640 1 -- start start 2023-11-02T16:13:26.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.620+0000 7f92cc59f640 10 monclient: init 2023-11-02T16:13:26.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:26.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2023-11-02T16:13:26.627 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92c40b3850 0x7f92c40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-c con 0x7f92c40b3850 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:26.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92c40ae8e0 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:26.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b0950 0x7f92c40b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-b con 0x7f92c40b0950 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:26.628 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c413f570 con 0x7f92c40ae8e0 2023-11-02T16:13:26.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.629 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c4141900 con 0x7f92c40b3850 2023-11-02T16:13:26.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.630 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4142800 con 0x7f92c40b0950 2023-11-02T16:13:26.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:26.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cc59f640 10 monclient(hunting): authenticate will time out at 3733.962909s 2023-11-02T16:13:26.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92cab3c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b0950 0x7f92c40b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92ca33b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92c40b3850 0x7f92c40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.631 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92c9b3a640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f92c40ae8e0 0x7f92c413ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:39804/0 (socket says 172.21.15.174:39804) 2023-11-02T16:13:26.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.621+0000 7f92c9b3a640 1 -- 172.21.15.174:0/4122695461 learned_addr learned my addr 172.21.15.174:0/4122695461 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:26.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): get_auth_request con 0x7f92c40b0950 auth_method 0 2023-11-02T16:13:26.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.632 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): get_auth_request con 0x7f92c40b3850 auth_method 0 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.633 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.634 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.622+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 10 monclient(hunting): handle_auth_done global_id 21514 payload 274 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 1 -- 172.21.15.174:0/4122695461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b0950 msgr2=0x7f92c40b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 1 --2- 172.21.15.174:0/4122695461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b0950 0x7f92c40b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 1 -- 172.21.15.174:0/4122695461 >> v1:172.21.15.174:6789/0 conn(0x7f92c40ae8e0 legacy=0x7f92c413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 10 monclient: _finish_hunting 0 2023-11-02T16:13:26.635 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 1 monclient: found mon.noname-c 2023-11-02T16:13:26.636 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:26.636 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:26.636 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.624+0000 7f92ca33b640 1 -- 172.21.15.174:0/4122695461 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40aed50 con 0x7f92c40b3850 2023-11-02T16:13:26.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.626+0000 7f92ca33b640 1 --2- 172.21.15.174:0/4122695461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40b3850 0x7f92c40b3c30 secure :-1 s=READY pgs=5260 cs=0 l=1 rev1=1 crypto rx=0x7f92b40097a0 tx=0x7f92b402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5723c3c87d8a63a server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 1 -- 172.21.15.174:0/4122695461 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b402e070 con 0x7f92c40b3850 2023-11-02T16:13:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: dump: 2023-11-02T16:13:26.643 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.643 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.643 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.643 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.643 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: _finish_auth 0 2023-11-02T16:13:26.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: _check_auth_tickets 2023-11-02T16:13:26.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:26.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 1 -- 172.21.15.174:0/4122695461 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40315a0 con 0x7f92c40b3850 2023-11-02T16:13:26.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92cc59f640 5 monclient: authenticate success, global_id 21514 2023-11-02T16:13:26.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92cc59f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:26.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:26.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92cc59f640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:26.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 1 -- 172.21.15.174:0/4122695461 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4031930 con 0x7f92c40b3850 2023-11-02T16:13:26.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92cc59f640 4 set_mon_vals no callback set 2023-11-02T16:13:26.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:26.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.631+0000 7f92c9339640 10 monclient: dump: 2023-11-02T16:13:26.647 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.647 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.647 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.648 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.648 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.648 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.648 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.648 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 10 monclient: shutdown 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 -- 172.21.15.174:0/4122695461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40b3850 msgr2=0x7f92c40b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 --2- 172.21.15.174:0/4122695461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40b3850 0x7f92c40b3c30 secure :-1 s=READY pgs=5260 cs=0 l=1 rev1=1 crypto rx=0x7f92b40097a0 tx=0x7f92b402b840 comp rx=0 tx=0).stop 2023-11-02T16:13:26.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 -- 172.21.15.174:0/4122695461 shutdown_connections 2023-11-02T16:13:26.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 --2- 172.21.15.174:0/4122695461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b0950 0x7f92c40b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 --2- 172.21.15.174:0/4122695461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40b3850 0x7f92c40b3c30 unknown :-1 s=CLOSED pgs=5260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 -- 172.21.15.174:0/4122695461 >> 172.21.15.174:0/4122695461 conn(0x7f92c4065960 msgr2=0x7f92c4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:26.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 -- 172.21.15.174:0/4122695461 shutdown_connections 2023-11-02T16:13:26.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 1 -- 172.21.15.174:0/4122695461 wait complete. 2023-11-02T16:13:26.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 10 monclient: build_initial_monmap 2023-11-02T16:13:26.652 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.632+0000 7f92cc59f640 10 monclient: monmap: 2023-11-02T16:13:26.652 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:26.652 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:26.652 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:26.652 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:26.653 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 librados: starting msgr at 2023-11-02T16:13:26.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 librados: starting objecter 2023-11-02T16:13:26.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 Processor -- start 2023-11-02T16:13:26.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 -- start start 2023-11-02T16:13:26.654 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 librados: setting wanted keys 2023-11-02T16:13:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 librados: calling monclient init 2023-11-02T16:13:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient: init 2023-11-02T16:13:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T16:13:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ae8e0 0x7f92c4139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-b con 0x7f92c40ae8e0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 0x7f92c4136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-a con 0x7f92c40b0950 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 0x7f92c4142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): picked mon.noname-c con 0x7f92c40b3850 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:26.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4136640 con 0x7f92c40b0950 2023-11-02T16:13:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41367b0 con 0x7f92c40ae8e0 2023-11-02T16:13:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41da330 con 0x7f92c40b3850 2023-11-02T16:13:26.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:26.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.633+0000 7f92cc59f640 10 monclient(hunting): authenticate will time out at 3733.975500s 2023-11-02T16:13:26.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92cab3c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 0x7f92c4142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 0x7f92c4136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92ca33b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ae8e0 0x7f92c4139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 0x7f92c4136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40810/0 (socket says 172.21.15.174:40810) 2023-11-02T16:13:26.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 1 -- 172.21.15.174:0/3837180080 learned_addr learned my addr 172.21.15.174:0/3837180080 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:26.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 10 monclient(hunting): get_auth_request con 0x7f92c40b0950 auth_method 0 2023-11-02T16:13:26.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92c9b3a640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92cab3c640 10 monclient(hunting): get_auth_request con 0x7f92c40b3850 auth_method 0 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92cab3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92cab3c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92cab3c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92ca33b640 10 monclient(hunting): get_auth_request con 0x7f92c40ae8e0 auth_method 0 2023-11-02T16:13:26.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92ca33b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:26.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92ca33b640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:26.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.634+0000 7f92ca33b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:26.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92c9b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92c9b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92c9b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.662 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92cab3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.663 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:26.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:26.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.635+0000 7f92ca33b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:26.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 10 monclient(hunting): handle_auth_done global_id 16916 payload 824 2023-11-02T16:13:26.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 -- 172.21.15.174:0/3837180080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ae8e0 msgr2=0x7f92c4139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ae8e0 0x7f92c4139f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 -- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 msgr2=0x7f92c4136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 0x7f92c4136c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 10 monclient: _finish_hunting 0 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 monclient: found mon.noname-c 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:26.667 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.637+0000 7f92cab3c640 1 -- 172.21.15.174:0/3837180080 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4009400 con 0x7f92c40b3850 2023-11-02T16:13:26.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.640+0000 7f92cab3c640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 0x7f92c4142bc0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00efc0 tx=0x7f92bc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.640+0000 7f92baffd640 10 client.?.objecter ms_handle_connect 0x7f92c40b3850 2023-11-02T16:13:26.668 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.640+0000 7f92baffd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:26.669 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc009c00 con 0x7f92c40b3850 2023-11-02T16:13:26.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:26.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: dump: 2023-11-02T16:13:26.670 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.671 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.671 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.671 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.672 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.672 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.672 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.672 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.672 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.673 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:26.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:26.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:26.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc00f040 con 0x7f92c40b3850 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.643+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0146c0 con 0x7f92c40b3850 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92baffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92baffd640 10 monclient: dump: 2023-11-02T16:13:26.674 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:26.675 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:26.675 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.675 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:26.675 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:26.675 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 5 monclient: authenticate success, global_id 16916 2023-11-02T16:13:26.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92c8b38640 4 set_mon_vals no callback set 2023-11-02T16:13:26.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _renew_subs 2023-11-02T16:13:26.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40af0b0 con 0x7f92c40b3850 2023-11-02T16:13:26.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 client.16916.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:26.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _renew_subs 2023-11-02T16:13:26.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c4135130 con 0x7f92c40b3850 2023-11-02T16:13:26.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 1 librados: init done 2023-11-02T16:13:26.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:26.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:26.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:26.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.644+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c40b4bf0 con 0x7f92c40b3850 2023-11-02T16:13:26.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.884+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f92bc014820 con 0x7f92c40b3850 2023-11-02T16:13:26.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.885+0000 7f92baffd640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 0x7f92a806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:26.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.885+0000 7f92ca33b640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 0x7f92a806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:26.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.885+0000 7f92ca33b640 10 monclient: get_auth_request con 0x7f92a806be10 auth_method 0 2023-11-02T16:13:26.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.886+0000 7f92baffd640 10 client.16916.objecter ms_handle_connect 0x7f92a806be10 2023-11-02T16:13:26.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.886+0000 7f92ca33b640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 0x7f92a806e2d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92b4036000 tx=0x7f92b403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:26.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f92bc058fc0 con 0x7f92c40b3850 2023-11-02T16:13:26.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 10 client.16916.objecter ms_dispatch 0x7f92c4000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:26.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 3 client.16916.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:26.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 3 client.16916.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:26.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 20 client.16916.objecter dump_active .. 0 homeless 2023-11-02T16:13:26.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f92bc055300 con 0x7f92c40b3850 2023-11-02T16:13:26.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:26.902 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:26.898+0000 7f92baffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:27.078 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.076+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c41da610 con 0x7f92a806be10 2023-11-02T16:13:27.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.077+0000 7f92baffd640 1 -- 172.21.15.174:0/3837180080 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f92c41da610 con 0x7f92a806be10 2023-11-02T16:13:27.079 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 0 to deep-scrub 2023-11-02T16:13:27.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 10 librados: watch_flush enter 2023-11-02T16:13:27.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 10 librados: watch_flush exit 2023-11-02T16:13:27.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 20 client.16916.objecter shutdown clearing up homeless session... 2023-11-02T16:13:27.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 10 client.16916.objecter successfully canceled tick 2023-11-02T16:13:27.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 msgr2=0x7f92a806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 0x7f92a806e2d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92b4036000 tx=0x7f92b403d040 comp rx=0 tx=0).stop 2023-11-02T16:13:27.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 10 monclient: shutdown 2023-11-02T16:13:27.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:27.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 msgr2=0x7f92c4142bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 0x7f92c4142bc0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00efc0 tx=0x7f92bc00c490 comp rx=0 tx=0).stop 2023-11-02T16:13:27.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 shutdown_connections 2023-11-02T16:13:27.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f92a806be10 0x7f92a806e2d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92c40b3850 0x7f92c4142bc0 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92c40b0950 0x7f92c4136c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 --2- 172.21.15.174:0/3837180080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c40ae8e0 0x7f92c4139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 >> 172.21.15.174:0/3837180080 conn(0x7f92c4065960 msgr2=0x7f92c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:27.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 shutdown_connections 2023-11-02T16:13:27.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 -- 172.21.15.174:0/3837180080 wait complete. 2023-11-02T16:13:27.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.079+0000 7f92cc59f640 1 librados: shutdown 2023-11-02T16:13:27.092 INFO:tasks.ceph:Scrubbing osd.1 2023-11-02T16:13:27.092 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:27.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 10 monclient: get_monmap_and_config 2023-11-02T16:13:27.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 10 monclient: build_initial_monmap 2023-11-02T16:13:27.195 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 10 monclient: monmap: 2023-11-02T16:13:27.195 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:27.195 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.196 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:27.194008+0000 2023-11-02T16:13:27.196 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:27.194008+0000 2023-11-02T16:13:27.196 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:27.196 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.196 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:27.197 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:27.197 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:27.197 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:27.197 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:27.197 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 1 Processor -- start 2023-11-02T16:13:27.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 1 -- start start 2023-11-02T16:13:27.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.193+0000 7f87d2c5f640 10 monclient: init 2023-11-02T16:13:27.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:27.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87cc0b3c10 0x7f87cc13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-a con 0x7f87cc0b3c10 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-b con 0x7f87cc13ede0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87cc0b0a00 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc0af970 con 0x7f87cc0b0a00 2023-11-02T16:13:27.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0b3ff0 con 0x7f87cc0b3c10 2023-11-02T16:13:27.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0afbc0 con 0x7f87cc13ede0 2023-11-02T16:13:27.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:27.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.194+0000 7f87d2c5f640 10 monclient(hunting): authenticate will time out at 3734.536096s 2023-11-02T16:13:27.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87cc0b3c10 0x7f87cc13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54622/0 (socket says 172.21.15.174:54622) 2023-11-02T16:13:27.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87cc0b3c10 0x7f87cc13e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40822/0 (socket says 172.21.15.174:40822) 2023-11-02T16:13:27.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 1 -- 172.21.15.174:0/534332272 learned_addr learned my addr 172.21.15.174:0/534332272 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:27.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 10 monclient(hunting): get_auth_request con 0x7f87cc13ede0 auth_method 0 2023-11-02T16:13:27.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87cbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 10 monclient(hunting): get_auth_request con 0x7f87cc0b3c10 auth_method 0 2023-11-02T16:13:27.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.195+0000 7f87d09fb640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.196+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 10 monclient(hunting): handle_auth_done global_id 15798 payload 274 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 1 -- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 msgr2=0x7f87cc0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 1 -- 172.21.15.174:0/534332272 >> v1:172.21.15.174:6789/0 conn(0x7f87cc0b0a00 legacy=0x7f87cc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:27.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 10 monclient: _finish_hunting 0 2023-11-02T16:13:27.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 1 monclient: found mon.noname-a 2023-11-02T16:13:27.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:27.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:27.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87d09fb640 1 -- 172.21.15.174:0/534332272 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0afea0 con 0x7f87cc0b3c10 2023-11-02T16:13:27.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.197+0000 7f87cbfff640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T16:13:27.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.200+0000 7f87d09fb640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc13e8a0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f87c00097a0 tx=0x7f87c002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07844db8e0402bf server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 1 -- 172.21.15.174:0/534332272 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c002f050 con 0x7f87cc0b3c10 2023-11-02T16:13:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 10 monclient: dump: 2023-11-02T16:13:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.214 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.214 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.214 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.214 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.214 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.215 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.215 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.215 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:27.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:27.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:27.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87d2c5f640 5 monclient: authenticate success, global_id 15798 2023-11-02T16:13:27.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.203+0000 7f87d2c5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:27.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 1 -- 172.21.15.174:0/534332272 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0031700 con 0x7f87cc0b3c10 2023-11-02T16:13:27.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:27.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 1 -- 172.21.15.174:0/534332272 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0031a90 con 0x7f87cc0b3c10 2023-11-02T16:13:27.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:27.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87cb7fe640 10 monclient: dump: 2023-11-02T16:13:27.217 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.218 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.218 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.218 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.218 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.219 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.219 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.219 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.219 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.219 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87d2c5f640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.204+0000 7f87d2c5f640 4 set_mon_vals no callback set 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 10 monclient: shutdown 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 -- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 msgr2=0x7f87cc13e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc13e8a0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f87c00097a0 tx=0x7f87c002b7a0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 -- 172.21.15.174:0/534332272 shutdown_connections 2023-11-02T16:13:27.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/534332272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc13e8a0 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 -- 172.21.15.174:0/534332272 >> 172.21.15.174:0/534332272 conn(0x7f87cc065960 msgr2=0x7f87cc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:27.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 -- 172.21.15.174:0/534332272 shutdown_connections 2023-11-02T16:13:27.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.205+0000 7f87d2c5f640 1 -- 172.21.15.174:0/534332272 wait complete. 2023-11-02T16:13:27.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 10 monclient: build_initial_monmap 2023-11-02T16:13:27.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 10 monclient: monmap: 2023-11-02T16:13:27.222 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:27.223 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:27.223 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:27.223 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:27.223 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:27.223 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 librados: starting msgr at 2023-11-02T16:13:27.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 librados: starting objecter 2023-11-02T16:13:27.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 Processor -- start 2023-11-02T16:13:27.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 -- start start 2023-11-02T16:13:27.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 librados: setting wanted keys 2023-11-02T16:13:27.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 1 librados: calling monclient init 2023-11-02T16:13:27.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.206+0000 7f87d2c5f640 10 monclient: init 2023-11-02T16:13:27.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:27.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T16:13:27.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87cc0b0a00 0x7f87cc139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-b con 0x7f87cc0b0a00 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:27.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc0a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-a con 0x7f87cc0b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:27.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): picked mon.noname-c con 0x7f87cc13ede0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:27.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc13bc80 con 0x7f87cc0b3c10 2023-11-02T16:13:27.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1d9eb0 con 0x7f87cc0b0a00 2023-11-02T16:13:27.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1da100 con 0x7f87cc13ede0 2023-11-02T16:13:27.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:27.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.207+0000 7f87d2c5f640 10 monclient(hunting): authenticate will time out at 3734.549238s 2023-11-02T16:13:27.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc0a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d11fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d09fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87cc0b0a00 0x7f87cc139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc0a8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40834/0 (socket says 172.21.15.174:40834) 2023-11-02T16:13:27.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 1 -- 172.21.15.174:0/1644408806 learned_addr learned my addr 172.21.15.174:0/1644408806 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:27.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 10 monclient(hunting): get_auth_request con 0x7f87cc0b3c10 auth_method 0 2023-11-02T16:13:27.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87cbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d11fc640 10 monclient(hunting): get_auth_request con 0x7f87cc13ede0 auth_method 0 2023-11-02T16:13:27.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d11fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d11fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d11fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d09fb640 10 monclient(hunting): get_auth_request con 0x7f87cc0b0a00 auth_method 0 2023-11-02T16:13:27.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d09fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d09fb640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.208+0000 7f87d09fb640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d09fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d11fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d11fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.209+0000 7f87d11fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.211+0000 7f87d11fc640 10 monclient(hunting): handle_auth_done global_id 16922 payload 824 2023-11-02T16:13:27.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 -- 172.21.15.174:0/1644408806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87cc0b0a00 msgr2=0x7f87cc139c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87cc0b0a00 0x7f87cc139c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 -- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 msgr2=0x7f87cc0a8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc0a8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 10 monclient: _finish_hunting 0 2023-11-02T16:13:27.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 monclient: found mon.noname-c 2023-11-02T16:13:27.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:27.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:27.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d11fc640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80094d0 con 0x7f87cc13ede0 2023-11-02T16:13:27.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87d09fb640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87cc0b0a00 0x7f87cc139c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T16:13:27.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.212+0000 7f87cbfff640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87cc0b3c10 0x7f87cc0a8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T16:13:27.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.214+0000 7f87d11fc640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0a98c0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c780 tx=0x7f87c400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.214+0000 7f87c97fa640 10 client.?.objecter ms_handle_connect 0x7f87cc13ede0 2023-11-02T16:13:27.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.214+0000 7f87c97fa640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:27.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.218+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003b70 con 0x7f87cc13ede0 2023-11-02T16:13:27.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.218+0000 7f87c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.218+0000 7f87c97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:27.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.218+0000 7f87c97fa640 10 monclient: dump: 2023-11-02T16:13:27.240 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.240 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.241 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.241 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.241 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.241 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.242 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.242 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.242 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.242 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: _finish_auth 0 2023-11-02T16:13:27.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: _check_auth_tickets 2023-11-02T16:13:27.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:27.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4003cf0 con 0x7f87cc13ede0 2023-11-02T16:13:27.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003ea0 con 0x7f87cc13ede0 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 5 monclient: authenticate success, global_id 16922 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 monclient: _renew_subs 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c0009400 con 0x7f87cc13ede0 2023-11-02T16:13:27.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87caffd640 4 set_mon_vals no callback set 2023-11-02T16:13:27.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:27.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87c97fa640 10 monclient: dump: 2023-11-02T16:13:27.245 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.245 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.246 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.246 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.246 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.246 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 client.16922.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:27.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 monclient: _renew_subs 2023-11-02T16:13:27.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:27.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc1da620 con 0x7f87cc13ede0 2023-11-02T16:13:27.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 1 librados: init done 2023-11-02T16:13:27.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 client.16922.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:27.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 15 client.16922.objecter _session_command_op_assign -1 1 2023-11-02T16:13:27.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 20 client.16922.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:27.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 client.16922.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:27.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:27.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.219+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87cc13f220 con 0x7f87cc13ede0 2023-11-02T16:13:27.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.240+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f87c40052b0 con 0x7f87cc13ede0 2023-11-02T16:13:27.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.240+0000 7f87c97fa640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f87b006be10 0x7f87b006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.240+0000 7f87d09fb640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f87b006be10 0x7f87b006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.240+0000 7f87d09fb640 10 monclient: get_auth_request con 0x7f87b006be10 auth_method 0 2023-11-02T16:13:27.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.241+0000 7f87c97fa640 10 client.16922.objecter ms_handle_connect 0x7f87b006be10 2023-11-02T16:13:27.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.241+0000 7f87d09fb640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f87b006be10 0x7f87b006e2d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f87c0035000 tx=0x7f87c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.243+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f87c4057ba0 con 0x7f87cc13ede0 2023-11-02T16:13:27.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.243+0000 7f87c97fa640 10 client.16922.objecter ms_dispatch 0x7f87cc000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:27.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.243+0000 7f87c97fa640 3 client.16922.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:27.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.243+0000 7f87c97fa640 3 client.16922.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:27.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 10 client.16922.objecter checking command 1 2023-11-02T16:13:27.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f87b0073780 0x7f87b0075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter _get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 15 client.16922.objecter _session_command_op_remove -1 1 2023-11-02T16:13:27.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter _get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 15 client.16922.objecter _session_command_op_assign 1 1 2023-11-02T16:13:27.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 4 2023-11-02T16:13:27.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 10 client.16922.objecter _send_command 1 2023-11-02T16:13:27.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87b00762f0 con 0x7f87b0073780 2023-11-02T16:13:27.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 20 client.16922.objecter dump_active .. 0 homeless 2023-11-02T16:13:27.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c4053ee0 con 0x7f87cc13ede0 2023-11-02T16:13:27.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87c97fa640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:27.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87cbfff640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f87b0073780 0x7f87b0075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.244+0000 7f87cbfff640 10 monclient: get_auth_request con 0x7f87b0073780 auth_method 0 2023-11-02T16:13:27.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.247+0000 7f87cbfff640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f87b0073780 0x7f87b0075c20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.247+0000 7f87c97fa640 10 client.16922.objecter ms_handle_connect 0x7f87b0073780 2023-11-02T16:13:27.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.273+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== osd.1 v2:172.21.15.174:6819/86146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f87b00762f0 con 0x7f87b0073780 2023-11-02T16:13:27.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.273+0000 7f87c97fa640 10 client.16922.objecter ms_dispatch 0x7f87cc000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:27.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.273+0000 7f87c97fa640 10 client.16922.objecter _finish_command 1 = system:0 2023-11-02T16:13:27.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.273+0000 7f87c97fa640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 4 2023-11-02T16:13:27.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.273+0000 7f87c97fa640 15 client.16922.objecter _session_command_op_remove 1 1 2023-11-02T16:13:27.287 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.285+0000 7f87d2c5f640 10 client.16922.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:27.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 15 client.16922.objecter _session_command_op_assign -1 2 2023-11-02T16:13:27.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 20 client.16922.objecter _get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 20 client.16922.objecter _get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 15 client.16922.objecter _session_command_op_remove -1 2 2023-11-02T16:13:27.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 20 client.16922.objecter get_session s=0x7f87b00730b0 osd=1 3 2023-11-02T16:13:27.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 15 client.16922.objecter _session_command_op_assign 1 2 2023-11-02T16:13:27.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 4 2023-11-02T16:13:27.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 10 client.16922.objecter _send_command 2 2023-11-02T16:13:27.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.286+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 --> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f87cc0ae380 con 0x7f87b0073780 2023-11-02T16:13:27.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.299+0000 7f87c97fa640 1 -- 172.21.15.174:0/1644408806 <== osd.1 v2:172.21.15.174:6819/86146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f87b8011560 con 0x7f87b0073780 2023-11-02T16:13:27.301 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.299+0000 7f87c97fa640 10 client.16922.objecter ms_dispatch 0x7f87cc000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:27.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.299+0000 7f87c97fa640 10 client.16922.objecter _finish_command 2 = system:0 2023-11-02T16:13:27.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.299+0000 7f87c97fa640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 4 2023-11-02T16:13:27.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.299+0000 7f87c97fa640 15 client.16922.objecter _session_command_op_remove 1 2 2023-11-02T16:13:27.302 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:27.303 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:27.303 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:27.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 10 librados: watch_flush enter 2023-11-02T16:13:27.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 10 librados: watch_flush exit 2023-11-02T16:13:27.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 10 client.16922.objecter close_session for osd.1 2023-11-02T16:13:27.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f87b0073780 msgr2=0x7f87b0075c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6819/86146,v1:172.21.15.174:6821/86146] conn(0x7f87b0073780 0x7f87b0075c20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 20 client.16922.objecter put_session s=0x7f87b00730b0 osd=1 1 2023-11-02T16:13:27.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 20 client.16922.objecter shutdown clearing up homeless session... 2023-11-02T16:13:27.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 10 client.16922.objecter successfully canceled tick 2023-11-02T16:13:27.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f87b006be10 msgr2=0x7f87b006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.306 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f87b006be10 0x7f87b006e2d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f87c0035000 tx=0x7f87c0008030 comp rx=0 tx=0).stop 2023-11-02T16:13:27.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 10 monclient: shutdown 2023-11-02T16:13:27.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:27.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 msgr2=0x7f87cc0a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 --2- 172.21.15.174:0/1644408806 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87cc13ede0 0x7f87cc0a98c0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c780 tx=0x7f87c400cc40 comp rx=0 tx=0).stop 2023-11-02T16:13:27.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d11fc640 1 -- 172.21.15.174:0/1644408806 reap_dead start 2023-11-02T16:13:27.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 shutdown_connections 2023-11-02T16:13:27.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 >> 172.21.15.174:0/1644408806 conn(0x7f87cc065960 msgr2=0x7f87cc073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:27.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 shutdown_connections 2023-11-02T16:13:27.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 -- 172.21.15.174:0/1644408806 wait complete. 2023-11-02T16:13:27.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.300+0000 7f87d2c5f640 1 librados: shutdown 2023-11-02T16:13:27.312 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-02T16:13:27.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.411+0000 7fd8f3f73640 10 monclient: get_monmap_and_config 2023-11-02T16:13:27.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.411+0000 7fd8f3f73640 10 monclient: build_initial_monmap 2023-11-02T16:13:27.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.411+0000 7fd8f3f73640 10 monclient: monmap: 2023-11-02T16:13:27.414 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:27.414 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.414 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:27.412725+0000 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:27.412725+0000 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:27.415 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:27.416 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:27.416 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:27.416 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.412+0000 7fd8f3f73640 1 Processor -- start 2023-11-02T16:13:27.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.412+0000 7fd8f3f73640 1 -- start start 2023-11-02T16:13:27.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.412+0000 7fd8f3f73640 10 monclient: init 2023-11-02T16:13:27.419 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:27.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2023-11-02T16:13:27.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 0x7fd8ec144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-b con 0x7fd8ec0b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:27.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8ec0b0ed0 0x7fd8ec0a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-c con 0x7fd8ec0b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:27.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8ec0b1cb0 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:27.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0aa1d0 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b4950 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0aa4b0 con 0x7fd8ec0b4570 2023-11-02T16:13:27.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:27.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f3f73640 10 monclient(hunting): authenticate will time out at 3734.754848s 2023-11-02T16:13:27.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f1d0f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 0x7fd8ec144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.413+0000 7fd8f150e640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8ec0b0ed0 0x7fd8ec0a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 0x7fd8ec144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54642/0 (socket says 172.21.15.174:54642) 2023-11-02T16:13:27.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 1 -- 172.21.15.174:0/2064561531 learned_addr learned my addr 172.21.15.174:0/2064561531 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:27.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): get_auth_request con 0x7fd8ec0b4570 auth_method 0 2023-11-02T16:13:27.423 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): get_auth_request con 0x7fd8ec0b0ed0 auth_method 0 2023-11-02T16:13:27.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.414+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 10 monclient(hunting): handle_auth_done global_id 21523 payload 274 2023-11-02T16:13:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 1 -- 172.21.15.174:0/2064561531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 msgr2=0x7fd8ec144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 1 --2- 172.21.15.174:0/2064561531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 0x7fd8ec144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 1 -- 172.21.15.174:0/2064561531 >> v1:172.21.15.174:6789/0 conn(0x7fd8ec0b1cb0 legacy=0x7fd8ec0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 10 monclient: _finish_hunting 0 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 1 monclient: found mon.noname-c 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:27.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.416+0000 7fd8f150e640 1 -- 172.21.15.174:0/2064561531 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0a8b50 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.418+0000 7fd8f150e640 1 --2- 172.21.15.174:0/2064561531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b0ed0 0x7fd8ec0a9070 secure :-1 s=READY pgs=5261 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009790 tx=0x7fd8dc02b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d41153482dd88 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 1 -- 172.21.15.174:0/2064561531 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc030710 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:27.433 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: dump: 2023-11-02T16:13:27.433 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.433 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.433 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.433 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.435 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: _finish_auth 0 2023-11-02T16:13:27.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: _check_auth_tickets 2023-11-02T16:13:27.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:27.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 1 -- 172.21.15.174:0/2064561531 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02e050 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:27.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 1 -- 172.21.15.174:0/2064561531 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc030a90 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:27.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f0d0d640 10 monclient: dump: 2023-11-02T16:13:27.437 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.437 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.437 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.437 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.437 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.438 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.438 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.438 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.438 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.438 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f3f73640 5 monclient: authenticate success, global_id 21523 2023-11-02T16:13:27.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f3f73640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:27.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.422+0000 7fd8f3f73640 4 set_mon_vals no callback set 2023-11-02T16:13:27.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 10 monclient: shutdown 2023-11-02T16:13:27.439 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:27.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 -- 172.21.15.174:0/2064561531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b0ed0 msgr2=0x7fd8ec0a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/2064561531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b0ed0 0x7fd8ec0a9070 secure :-1 s=READY pgs=5261 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009790 tx=0x7fd8dc02b6f0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 -- 172.21.15.174:0/2064561531 shutdown_connections 2023-11-02T16:13:27.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/2064561531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b0ed0 0x7fd8ec0a9070 unknown :-1 s=CLOSED pgs=5261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/2064561531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b4570 0x7fd8ec144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 -- 172.21.15.174:0/2064561531 >> 172.21.15.174:0/2064561531 conn(0x7fd8ec065d50 msgr2=0x7fd8ec066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:27.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 -- 172.21.15.174:0/2064561531 shutdown_connections 2023-11-02T16:13:27.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.423+0000 7fd8f3f73640 1 -- 172.21.15.174:0/2064561531 wait complete. 2023-11-02T16:13:27.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 10 monclient: build_initial_monmap 2023-11-02T16:13:27.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 10 monclient: monmap: 2023-11-02T16:13:27.442 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:27.442 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:27.442 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:27.442 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:27.442 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 librados: starting msgr at 2023-11-02T16:13:27.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 librados: starting objecter 2023-11-02T16:13:27.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 Processor -- start 2023-11-02T16:13:27.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 -- start start 2023-11-02T16:13:27.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 librados: setting wanted keys 2023-11-02T16:13:27.444 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 1 librados: calling monclient init 2023-11-02T16:13:27.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.424+0000 7fd8f3f73640 10 monclient: init 2023-11-02T16:13:27.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:27.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient: _add_conns ranks=[2,1,0] 2023-11-02T16:13:27.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b0ed0 0x7fd8ec140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-c con 0x7fd8ec0b0ed0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:27.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-b con 0x7fd8ec0b1cb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:27.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8ec0b4570 0x7fd8ec138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): picked mon.noname-a con 0x7fd8ec0b4570 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:27.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0ac740 con 0x7fd8ec0b4570 2023-11-02T16:13:27.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec1d6870 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:27.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec1d6520 con 0x7fd8ec0b0ed0 2023-11-02T16:13:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.425+0000 7fd8f3f73640 10 monclient(hunting): authenticate will time out at 3734.767279s 2023-11-02T16:13:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f1d0f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b0ed0 0x7fd8ec140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f2510640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8ec0b4570 0x7fd8ec138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54812/0 (socket says 172.21.15.174:54812) 2023-11-02T16:13:27.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 1 -- 172.21.15.174:0/3196843475 learned_addr learned my addr 172.21.15.174:0/3196843475 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:27.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): get_auth_request con 0x7fd8ec0b1cb0 auth_method 0 2023-11-02T16:13:27.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.449 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f1d0f640 10 monclient(hunting): get_auth_request con 0x7fd8ec0b0ed0 auth_method 0 2023-11-02T16:13:27.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f1d0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f1d0f640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.450 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f1d0f640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f150e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f2510640 10 monclient(hunting): get_auth_request con 0x7fd8ec0b4570 auth_method 0 2023-11-02T16:13:27.451 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f2510640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f2510640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.426+0000 7fd8f2510640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f1d0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.452 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f2510640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:27.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f2510640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:27.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.427+0000 7fd8f2510640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:27.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 10 monclient(hunting): handle_auth_done global_id 21526 payload 824 2023-11-02T16:13:27.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 -- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b0ed0 msgr2=0x7fd8ec140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.453 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b0ed0 0x7fd8ec140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 -- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8ec0b4570 msgr2=0x7fd8ec138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8ec0b4570 0x7fd8ec138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 10 monclient: _finish_hunting 0 2023-11-02T16:13:27.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 monclient: found mon.noname-b 2023-11-02T16:13:27.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:27.455 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:27.455 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.429+0000 7fd8f150e640 1 -- 172.21.15.174:0/3196843475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0093a0 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.432+0000 7fd8f150e640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 secure :-1 s=READY pgs=5262 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.456 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.432+0000 7fd8e27fc640 10 client.?.objecter ms_handle_connect 0x7fd8ec0b1cb0 2023-11-02T16:13:27.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.432+0000 7fd8e27fc640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:27.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc03b810 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.457 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:27.458 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: dump: 2023-11-02T16:13:27.458 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.458 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.458 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.458 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.459 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.459 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.459 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.459 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.459 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.460 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:27.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:27.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:27.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc03b990 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:27.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc03bd40 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 5 monclient: authenticate success, global_id 21526 2023-11-02T16:13:27.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 10 monclient: _renew_subs 2023-11-02T16:13:27.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:27.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec1d72d0 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e3fff640 4 set_mon_vals no callback set 2023-11-02T16:13:27.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:27.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:27.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8e27fc640 10 monclient: dump: 2023-11-02T16:13:27.463 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:27.463 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.463 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.463 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:27.463 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:27.464 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.464 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:27.464 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:27.464 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:27.464 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 10 client.21526.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 10 monclient: _renew_subs 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec138d70 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.435+0000 7fd8f3f73640 1 librados: init done 2023-11-02T16:13:27.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.436+0000 7fd8f3f73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:27.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.436+0000 7fd8f3f73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:27.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.436+0000 7fd8f3f73640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:27.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.436+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec1d66e0 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.677+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd8dc039040 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.677+0000 7fd8e27fc640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 0x7fd8d006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.677+0000 7fd8f1d0f640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 0x7fd8d006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.677+0000 7fd8f1d0f640 10 monclient: get_auth_request con 0x7fd8d006bdc0 auth_method 0 2023-11-02T16:13:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.678+0000 7fd8e27fc640 10 client.21526.objecter ms_handle_connect 0x7fd8d006bdc0 2023-11-02T16:13:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.678+0000 7fd8f1d0f640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 0x7fd8d006e280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b30 tx=0x7fd8e8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.679+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fd8dc079950 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.679+0000 7fd8e27fc640 10 client.21526.objecter ms_dispatch 0x7fd8ec000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.679+0000 7fd8e27fc640 3 client.21526.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.679+0000 7fd8e27fc640 3 client.21526.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:27.682 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.679+0000 7fd8e27fc640 20 client.21526.objecter dump_active .. 0 homeless 2023-11-02T16:13:27.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.689+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd8dc075c90 con 0x7fd8ec0b1cb0 2023-11-02T16:13:27.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.689+0000 7fd8e27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:27.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.689+0000 7fd8e27fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:27.877 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.873+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8ec0a95b0 con 0x7fd8d006bdc0 2023-11-02T16:13:27.877 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.873+0000 7fd8e27fc640 1 -- 172.21.15.174:0/3196843475 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd8e8011a60 con 0x7fd8d006bdc0 2023-11-02T16:13:27.877 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 1 to deep-scrub 2023-11-02T16:13:27.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 10 librados: watch_flush enter 2023-11-02T16:13:27.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 10 librados: watch_flush exit 2023-11-02T16:13:27.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 20 client.21526.objecter shutdown clearing up homeless session... 2023-11-02T16:13:27.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 10 client.21526.objecter successfully canceled tick 2023-11-02T16:13:27.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 msgr2=0x7fd8d006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 0x7fd8d006e280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b30 tx=0x7fd8e8009240 comp rx=0 tx=0).stop 2023-11-02T16:13:27.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 10 monclient: shutdown 2023-11-02T16:13:27.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:27.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 msgr2=0x7fd8ec1d6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:27.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 secure :-1 s=READY pgs=5262 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc0046c0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 shutdown_connections 2023-11-02T16:13:27.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8ec0b4570 0x7fd8ec138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8ec0b1cb0 0x7fd8ec1d6d90 unknown :-1 s=CLOSED pgs=5262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fd8d006bdc0 0x7fd8d006e280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 --2- 172.21.15.174:0/3196843475 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8ec0b0ed0 0x7fd8ec140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:27.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 >> 172.21.15.174:0/3196843475 conn(0x7fd8ec065d50 msgr2=0x7fd8ec13ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:27.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 shutdown_connections 2023-11-02T16:13:27.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 -- 172.21.15.174:0/3196843475 wait complete. 2023-11-02T16:13:27.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.876+0000 7fd8f3f73640 1 librados: shutdown 2023-11-02T16:13:27.889 INFO:tasks.ceph:Scrubbing osd.2 2023-11-02T16:13:27.889 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:27.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 10 monclient: get_monmap_and_config 2023-11-02T16:13:27.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 10 monclient: build_initial_monmap 2023-11-02T16:13:27.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 10 monclient: monmap: 2023-11-02T16:13:27.996 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:27.996 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:27.997 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:27.995211+0000 2023-11-02T16:13:27.997 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:27.995211+0000 2023-11-02T16:13:27.997 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:27.997 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:27.997 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:27.998 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:27.998 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:27.998 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:27.998 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:27.998 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 1 Processor -- start 2023-11-02T16:13:28.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 1 -- start start 2023-11-02T16:13:28.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.994+0000 7f93b50e5640 10 monclient: init 2023-11-02T16:13:28.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2023-11-02T16:13:28.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 0x7f93b013e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-b con 0x7f93b00b3c10 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:28.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f93b013ede0 0x7f93b00b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-a con 0x7f93b013ede0 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:28.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f93b00b0a00 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:28.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b00af970 con 0x7f93b00b0a00 2023-11-02T16:13:28.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b00b3ff0 con 0x7f93b013ede0 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00afbc0 con 0x7f93b00b3c10 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.995+0000 7f93b50e5640 10 monclient(hunting): authenticate will time out at 3735.337240s 2023-11-02T16:13:28.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93ae59c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f93b013ede0 0x7f93b00b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93aed9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 0x7f93b013e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93af59e640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f93b00b0a00 0x7f93b00b1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51384/0 (socket says 172.21.15.174:51384) 2023-11-02T16:13:28.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93af59e640 1 -- 172.21.15.174:0/2957279902 learned_addr learned my addr 172.21.15.174:0/2957279902 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:28.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93ae59c640 10 monclient(hunting): get_auth_request con 0x7f93b013ede0 auth_method 0 2023-11-02T16:13:28.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93ae59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93ae59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93aed9d640 10 monclient(hunting): get_auth_request con 0x7f93b00b3c10 auth_method 0 2023-11-02T16:13:28.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93aed9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.996+0000 7f93aed9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.997+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 1 -- 172.21.15.174:0/2957279902 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218281437 0 0) 0x7f93b00af970 con 0x7f93b00b0a00 2023-11-02T16:13:28.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 10 monclient(hunting): _negotiate 2023-11-02T16:13:28.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 10 monclient(hunting): my global_id is 21529 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93add9b640 1 -- 172.21.15.174:0/2957279902 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9398003520 con 0x7f93b00b0a00 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 10 monclient(hunting): handle_auth_done global_id 16931 payload 274 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 1 -- 172.21.15.174:0/2957279902 >> v2:172.21.15.174:3300/0 conn(0x7f93b013ede0 msgr2=0x7f93b00b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:28.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 1 --2- 172.21.15.174:0/2957279902 >> v2:172.21.15.174:3300/0 conn(0x7f93b013ede0 0x7f93b00b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 1 -- 172.21.15.174:0/2957279902 >> v1:172.21.15.175:6789/0 conn(0x7f93b00b0a00 legacy=0x7f93b00b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 1 monclient: found mon.noname-b 2023-11-02T16:13:28.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:28.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:27.998+0000 7f93aed9d640 1 -- 172.21.15.174:0/2957279902 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00afea0 con 0x7f93b00b3c10 2023-11-02T16:13:28.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.000+0000 7f93aed9d640 1 --2- 172.21.15.174:0/2957279902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 0x7f93b013e8a0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097a0 tx=0x7f93a402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b333b97b45248b server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.015 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 1 -- 172.21.15.174:0/2957279902 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a402e070 con 0x7f93b00b3c10 2023-11-02T16:13:28.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:28.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 10 monclient: dump: 2023-11-02T16:13:28.016 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.016 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.017 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.017 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.017 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.017 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.017 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.018 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.018 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.018 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.019 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93add9b640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.019 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93b50e5640 5 monclient: authenticate success, global_id 16931 2023-11-02T16:13:28.019 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.003+0000 7f93b50e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:28.019 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 1 -- 172.21.15.174:0/2957279902 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a40315a0 con 0x7f93b00b3c10 2023-11-02T16:13:28.019 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 1 -- 172.21.15.174:0/2957279902 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4031910 con 0x7f93b00b3c10 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93add9b640 10 monclient: dump: 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.020 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.021 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.021 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.021 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.021 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.021 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.022 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.022 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.022 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.022 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93b50e5640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:28.022 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.004+0000 7f93b50e5640 4 set_mon_vals no callback set 2023-11-02T16:13:28.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 10 monclient: shutdown 2023-11-02T16:13:28.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:28.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 -- 172.21.15.174:0/2957279902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 msgr2=0x7f93b013e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 --2- 172.21.15.174:0/2957279902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 0x7f93b013e8a0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097a0 tx=0x7f93a402b840 comp rx=0 tx=0).stop 2023-11-02T16:13:28.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 -- 172.21.15.174:0/2957279902 shutdown_connections 2023-11-02T16:13:28.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 --2- 172.21.15.174:0/2957279902 >> v2:172.21.15.174:3300/0 conn(0x7f93b013ede0 0x7f93b00b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 --2- 172.21.15.174:0/2957279902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b00b3c10 0x7f93b013e8a0 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 -- 172.21.15.174:0/2957279902 >> 172.21.15.174:0/2957279902 conn(0x7f93b0065960 msgr2=0x7f93b0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:28.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 -- 172.21.15.174:0/2957279902 shutdown_connections 2023-11-02T16:13:28.024 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.005+0000 7f93b50e5640 1 -- 172.21.15.174:0/2957279902 wait complete. 2023-11-02T16:13:28.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 10 monclient: build_initial_monmap 2023-11-02T16:13:28.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 10 monclient: monmap: 2023-11-02T16:13:28.025 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:28.025 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:28.025 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:28.026 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:28.026 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:28.026 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.026 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:28.026 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 librados: starting msgr at 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 librados: starting objecter 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 Processor -- start 2023-11-02T16:13:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 -- start start 2023-11-02T16:13:28.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 librados: setting wanted keys 2023-11-02T16:13:28.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 1 librados: calling monclient init 2023-11-02T16:13:28.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.006+0000 7f93b50e5640 10 monclient: init 2023-11-02T16:13:28.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T16:13:28.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b00b0a00 0x7f93b01d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-a con 0x7f93b00b0a00 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:28.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b00b3c10 0x7f93b013ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-b con 0x7f93b00b3c10 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:28.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b013ede0 0x7f93b013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): picked mon.noname-c con 0x7f93b013ede0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:28.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b01d66e0 con 0x7f93b00b0a00 2023-11-02T16:13:28.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b013a0d0 con 0x7f93b00b3c10 2023-11-02T16:13:28.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0139420 con 0x7f93b013ede0 2023-11-02T16:13:28.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.007+0000 7f93b50e5640 10 monclient(hunting): authenticate will time out at 3735.349159s 2023-11-02T16:13:28.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93aed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b00b0a00 0x7f93b01d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b013ede0 0x7f93b013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93ae59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b00b3c10 0x7f93b013ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b013ede0 0x7f93b013b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54662/0 (socket says 172.21.15.174:54662) 2023-11-02T16:13:28.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 1 -- 172.21.15.174:0/1370039507 learned_addr learned my addr 172.21.15.174:0/1370039507 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:28.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): get_auth_request con 0x7f93b013ede0 auth_method 0 2023-11-02T16:13:28.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93aed9d640 10 monclient(hunting): get_auth_request con 0x7f93b00b0a00 auth_method 0 2023-11-02T16:13:28.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93aed9d640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93aed9d640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93ae59c640 10 monclient(hunting): get_auth_request con 0x7f93b00b3c10 auth_method 0 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93ae59c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93ae59c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.008+0000 7f93af59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93aed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.009+0000 7f93ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.036 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 10 monclient(hunting): handle_auth_done global_id 21532 payload 824 2023-11-02T16:13:28.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 -- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b013ede0 msgr2=0x7f93b013b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93b013ede0 0x7f93b013b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 -- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b00b0a00 msgr2=0x7f93b01d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b00b0a00 0x7f93b01d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 monclient: found mon.noname-b 2023-11-02T16:13:28.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.011+0000 7f93ae59c640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4009400 con 0x7f93b00b3c10 2023-11-02T16:13:28.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.014+0000 7f93ae59c640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b00b3c10 0x7f93b013ab00 secure :-1 s=READY pgs=5264 cs=0 l=1 rev1=1 crypto rx=0x7f93a000e870 tx=0x7f93a000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.014+0000 7f93977fe640 10 client.?.objecter ms_handle_connect 0x7f93b00b3c10 2023-11-02T16:13:28.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.014+0000 7f93977fe640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:28.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000ee60 con 0x7f93b00b3c10 2023-11-02T16:13:28.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:28.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: dump: 2023-11-02T16:13:28.042 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.042 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.042 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.042 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.043 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.043 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.043 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.043 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.043 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.044 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.044 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000c940 con 0x7f93b00b3c10 2023-11-02T16:13:28.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 5 monclient: authenticate success, global_id 21532 2023-11-02T16:13:28.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 10 monclient: _renew_subs 2023-11-02T16:13:28.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b0143200 con 0x7f93b00b3c10 2023-11-02T16:13:28.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000ccd0 con 0x7f93b00b3c10 2023-11-02T16:13:28.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 10 client.21532.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:28.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:28.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93977fe640 10 monclient: dump: 2023-11-02T16:13:28.047 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.047 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.047 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.047 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.047 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 10 monclient: _renew_subs 2023-11-02T16:13:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b01d7320 con 0x7f93b00b3c10 2023-11-02T16:13:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93b50e5640 1 librados: init done 2023-11-02T16:13:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.017+0000 7f93ad59a640 4 set_mon_vals no callback set 2023-11-02T16:13:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 10 client.21532.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:28.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 15 client.21532.objecter _session_command_op_assign -1 1 2023-11-02T16:13:28.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 20 client.21532.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:28.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 10 client.21532.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:28.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.018+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93b013f220 con 0x7f93b00b3c10 2023-11-02T16:13:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f93a0013470 con 0x7f93b00b3c10 2023-11-02T16:13:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93977fe640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f939c06bdc0 0x7f939c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93aed9d640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f939c06bdc0 0x7f939c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93aed9d640 10 monclient: get_auth_request con 0x7f939c06bdc0 auth_method 0 2023-11-02T16:13:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93aed9d640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f939c06bdc0 0x7f939c06e280 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93a4036000 tx=0x7f93a4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.039+0000 7f93977fe640 10 client.21532.objecter ms_handle_connect 0x7f939c06bdc0 2023-11-02T16:13:28.053 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f93a000ce30 con 0x7f93b00b3c10 2023-11-02T16:13:28.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 10 client.21532.objecter ms_dispatch 0x7f93b0000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:28.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 3 client.21532.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:28.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 3 client.21532.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:28.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 10 client.21532.objecter checking command 1 2023-11-02T16:13:28.054 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f939c073730 0x7f939c075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter _get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 15 client.21532.objecter _session_command_op_remove -1 1 2023-11-02T16:13:28.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter _get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.055 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 15 client.21532.objecter _session_command_op_assign 2 1 2023-11-02T16:13:28.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 4 2023-11-02T16:13:28.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 10 client.21532.objecter _send_command 1 2023-11-02T16:13:28.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f939c0762a0 con 0x7f939c073730 2023-11-02T16:13:28.056 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93977fe640 20 client.21532.objecter dump_active .. 0 homeless 2023-11-02T16:13:28.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.042+0000 7f93af59e640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f939c073730 0x7f939c075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.043+0000 7f93af59e640 10 monclient: get_auth_request con 0x7f939c073730 auth_method 0 2023-11-02T16:13:28.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.043+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a0054290 con 0x7f93b00b3c10 2023-11-02T16:13:28.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.043+0000 7f93977fe640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:28.057 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.045+0000 7f93af59e640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f939c073730 0x7f939c075bd0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.058 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.045+0000 7f93977fe640 10 client.21532.objecter ms_handle_connect 0x7f939c073730 2023-11-02T16:13:28.069 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.067+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== osd.2 v2:172.21.15.174:6810/86153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f939c0762a0 con 0x7f939c073730 2023-11-02T16:13:28.069 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.067+0000 7f93977fe640 10 client.21532.objecter ms_dispatch 0x7f93b0000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:28.069 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.067+0000 7f93977fe640 10 client.21532.objecter _finish_command 1 = system:0 2023-11-02T16:13:28.069 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.067+0000 7f93977fe640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 4 2023-11-02T16:13:28.070 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.067+0000 7f93977fe640 15 client.21532.objecter _session_command_op_remove 2 1 2023-11-02T16:13:28.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 10 client.21532.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:28.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 15 client.21532.objecter _session_command_op_assign -1 2 2023-11-02T16:13:28.081 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 20 client.21532.objecter _get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 20 client.21532.objecter _get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 15 client.21532.objecter _session_command_op_remove -1 2 2023-11-02T16:13:28.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 20 client.21532.objecter get_session s=0x7f939c073060 osd=2 3 2023-11-02T16:13:28.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 15 client.21532.objecter _session_command_op_assign 2 2 2023-11-02T16:13:28.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 4 2023-11-02T16:13:28.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 10 client.21532.objecter _send_command 2 2023-11-02T16:13:28.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.079+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 --> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f93b00ae380 con 0x7f939c073730 2023-11-02T16:13:28.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93977fe640 1 -- 172.21.15.174:0/1370039507 <== osd.2 v2:172.21.15.174:6810/86153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f93a8001fe0 con 0x7f939c073730 2023-11-02T16:13:28.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93977fe640 10 client.21532.objecter ms_dispatch 0x7f93b0000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:28.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93977fe640 10 client.21532.objecter _finish_command 2 = system:0 2023-11-02T16:13:28.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93977fe640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 4 2023-11-02T16:13:28.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93977fe640 15 client.21532.objecter _session_command_op_remove 2 2 2023-11-02T16:13:28.096 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:28.096 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:28.097 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:28.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 10 librados: watch_flush enter 2023-11-02T16:13:28.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 10 librados: watch_flush exit 2023-11-02T16:13:28.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 10 client.21532.objecter close_session for osd.2 2023-11-02T16:13:28.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f939c073730 msgr2=0x7f939c075bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6810/86153,v1:172.21.15.174:6811/86153] conn(0x7f939c073730 0x7f939c075bd0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 20 client.21532.objecter put_session s=0x7f939c073060 osd=2 1 2023-11-02T16:13:28.101 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 20 client.21532.objecter shutdown clearing up homeless session... 2023-11-02T16:13:28.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 10 client.21532.objecter successfully canceled tick 2023-11-02T16:13:28.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f939c06bdc0 msgr2=0x7f939c06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f939c06bdc0 0x7f939c06e280 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93a4036000 tx=0x7f93a4008030 comp rx=0 tx=0).stop 2023-11-02T16:13:28.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 10 monclient: shutdown 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b00b3c10 msgr2=0x7f93b013ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 --2- 172.21.15.174:0/1370039507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93b00b3c10 0x7f93b013ab00 secure :-1 s=READY pgs=5264 cs=0 l=1 rev1=1 crypto rx=0x7f93a000e870 tx=0x7f93a000ed30 comp rx=0 tx=0).stop 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93af59e640 1 -- 172.21.15.174:0/1370039507 reap_dead start 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 shutdown_connections 2023-11-02T16:13:28.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 >> 172.21.15.174:0/1370039507 conn(0x7f93b0065960 msgr2=0x7f93b0073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:28.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 shutdown_connections 2023-11-02T16:13:28.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.093+0000 7f93b50e5640 1 -- 172.21.15.174:0/1370039507 wait complete. 2023-11-02T16:13:28.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.094+0000 7f93b50e5640 1 librados: shutdown 2023-11-02T16:13:28.106 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-02T16:13:28.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.205+0000 7fe701d0a640 10 monclient: get_monmap_and_config 2023-11-02T16:13:28.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.205+0000 7fe701d0a640 10 monclient: build_initial_monmap 2023-11-02T16:13:28.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.205+0000 7fe701d0a640 10 monclient: monmap: 2023-11-02T16:13:28.210 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:28.210 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:28.206425+0000 2023-11-02T16:13:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:28.206425+0000 2023-11-02T16:13:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:28.212 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:28.212 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:28.212 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:28.212 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:28.212 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 Processor -- start 2023-11-02T16:13:28.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 -- start start 2023-11-02T16:13:28.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient: init 2023-11-02T16:13:28.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient: _add_conns ranks=[0,4,1,3,2] 2023-11-02T16:13:28.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6fc06b7d0 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:28.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe6fc0ae1c0 0x7fe6fc141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-c con 0x7fe6fc0ae1c0 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:28.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6fc142250 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:28.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc06bbb0 con 0x7fe6fc06b7d0 2023-11-02T16:13:28.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc0ad0d0 con 0x7fe6fc142250 2023-11-02T16:13:28.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.215 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc0af5b0 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.206+0000 7fe701d0a640 10 monclient(hunting): authenticate will time out at 3735.548499s 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.207+0000 7fe6fb7fe640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fe6fc06b7d0 0x7fe6fc0adab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:39836/0 (socket says 172.21.15.174:39836) 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.207+0000 7fe6fb7fe640 1 -- 172.21.15.174:0/3620358073 learned_addr learned my addr 172.21.15.174:0/3620358073 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.207+0000 7fe6faffd640 1 --2- 172.21.15.174:0/3620358073 >> v2:172.21.15.175:3300/0 conn(0x7fe6fc0ae1c0 0x7fe6fc141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): get_auth_request con 0x7fe6fc0ae1c0 auth_method 0 2023-11-02T16:13:28.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.208+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 1 -- 172.21.15.174:0/3620358073 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23059697 0 0) 0x7fe6fc06bbb0 con 0x7fe6fc06b7d0 2023-11-02T16:13:28.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 10 monclient(hunting): _negotiate 2023-11-02T16:13:28.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 10 monclient(hunting): my global_id is 15813 2023-11-02T16:13:28.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.209+0000 7fe6fa7fc640 1 -- 172.21.15.174:0/3620358073 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e4003520 con 0x7fe6fc06b7d0 2023-11-02T16:13:28.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 10 monclient(hunting): handle_auth_done global_id 21535 payload 274 2023-11-02T16:13:28.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 1 -- 172.21.15.174:0/3620358073 >> v1:172.21.15.175:6789/0 conn(0x7fe6fc142250 legacy=0x7fe6fc14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 1 -- 172.21.15.174:0/3620358073 >> v1:172.21.15.174:6789/0 conn(0x7fe6fc06b7d0 legacy=0x7fe6fc0adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 1 monclient: found mon.noname-c 2023-11-02T16:13:28.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.210+0000 7fe6faffd640 1 -- 172.21.15.174:0/3620358073 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0ad3b0 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.214+0000 7fe6faffd640 1 --2- 172.21.15.174:0/3620358073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc141d10 secure :-1 s=READY pgs=5266 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00097a0 tx=0x7fe6f002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6af1a7821293a785 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 1 -- 172.21.15.174:0/3620358073 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0031830 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:28.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: dump: 2023-11-02T16:13:28.228 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.229 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.229 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.229 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.229 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.230 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.230 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.230 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.230 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.230 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 1 -- 172.21.15.174:0/3620358073 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0031990 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe701d0a640 5 monclient: authenticate success, global_id 21535 2023-11-02T16:13:28.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe701d0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:28.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 1 -- 172.21.15.174:0/3620358073 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0031d20 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe701d0a640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:28.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe701d0a640 4 set_mon_vals no callback set 2023-11-02T16:13:28.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:28.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.219+0000 7fe6fa7fc640 10 monclient: dump: 2023-11-02T16:13:28.234 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.234 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.234 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.234 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.234 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.235 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.235 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.235 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.235 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.235 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 10 monclient: shutdown 2023-11-02T16:13:28.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:28.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 -- 172.21.15.174:0/3620358073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc0ae1c0 msgr2=0x7fe6fc141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 --2- 172.21.15.174:0/3620358073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc141d10 secure :-1 s=READY pgs=5266 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00097a0 tx=0x7fe6f002b8d0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 -- 172.21.15.174:0/3620358073 shutdown_connections 2023-11-02T16:13:28.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 --2- 172.21.15.174:0/3620358073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc141d10 unknown :-1 s=CLOSED pgs=5266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 -- 172.21.15.174:0/3620358073 >> 172.21.15.174:0/3620358073 conn(0x7fe6fc0b5060 msgr2=0x7fe6fc0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:28.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 -- 172.21.15.174:0/3620358073 shutdown_connections 2023-11-02T16:13:28.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 1 -- 172.21.15.174:0/3620358073 wait complete. 2023-11-02T16:13:28.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 10 monclient: build_initial_monmap 2023-11-02T16:13:28.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.220+0000 7fe701d0a640 10 monclient: monmap: 2023-11-02T16:13:28.238 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:28.238 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:28.238 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:28.239 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:28.239 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:28.239 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.239 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:28.240 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:28.240 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:28.240 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 librados: starting msgr at 2023-11-02T16:13:28.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 librados: starting objecter 2023-11-02T16:13:28.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 Processor -- start 2023-11-02T16:13:28.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 -- start start 2023-11-02T16:13:28.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 librados: setting wanted keys 2023-11-02T16:13:28.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 librados: calling monclient init 2023-11-02T16:13:28.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient: init 2023-11-02T16:13:28.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T16:13:28.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6fc06b7d0 0x7fe6fc1da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-c con 0x7fe6fc06b7d0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:28.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc11d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-a con 0x7fe6fc0ae1c0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:28.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): picked mon.noname-b con 0x7fe6fc142250 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:28.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc11d660 con 0x7fe6fc0ae1c0 2023-11-02T16:13:28.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc11d880 con 0x7fe6fc142250 2023-11-02T16:13:28.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc143c80 con 0x7fe6fc06b7d0 2023-11-02T16:13:28.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.221+0000 7fe701d0a640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe701d0a640 10 monclient(hunting): authenticate will time out at 3735.563656s 2023-11-02T16:13:28.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc11d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6fc06b7d0 0x7fe6fc1da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54844/0 (socket says 172.21.15.174:54844) 2023-11-02T16:13:28.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 1 -- 172.21.15.174:0/2407048871 learned_addr learned my addr 172.21.15.174:0/2407048871 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:28.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 10 monclient(hunting): get_auth_request con 0x7fe6fc142250 auth_method 0 2023-11-02T16:13:28.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6fbfff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6faffd640 10 monclient(hunting): get_auth_request con 0x7fe6fc0ae1c0 auth_method 0 2023-11-02T16:13:28.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6faffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.222+0000 7fe6faffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): get_auth_request con 0x7fe6fc06b7d0 auth_method 0 2023-11-02T16:13:28.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.223+0000 7fe6fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 10 monclient(hunting): handle_auth_done global_id 21541 payload 824 2023-11-02T16:13:28.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 -- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6fc06b7d0 msgr2=0x7fe6fc1da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6fc06b7d0 0x7fe6fc1da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 -- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6fc0ae1c0 msgr2=0x7fe6fc11d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc11d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 monclient: found mon.noname-b 2023-11-02T16:13:28.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.226+0000 7fe6fbfff640 1 -- 172.21.15.174:0/2407048871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009400 con 0x7fe6fc142250 2023-11-02T16:13:28.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.228+0000 7fe6fbfff640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 secure :-1 s=READY pgs=5267 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c6e0 tx=0x7fe6f400cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.228+0000 7fe6e3fff640 10 client.?.objecter ms_handle_connect 0x7fe6fc142250 2023-11-02T16:13:28.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.228+0000 7fe6e3fff640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:28.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003b70 con 0x7fe6fc142250 2023-11-02T16:13:28.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:28.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: dump: 2023-11-02T16:13:28.260 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.260 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.260 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.260 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.261 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.261 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.261 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.261 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.261 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4003cf0 con 0x7fe6fc142250 2023-11-02T16:13:28.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 5 monclient: authenticate success, global_id 21541 2023-11-02T16:13:28.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 10 monclient: _renew_subs 2023-11-02T16:13:28.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc1dacd0 con 0x7fe6fc142250 2023-11-02T16:13:28.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4005310 con 0x7fe6fc142250 2023-11-02T16:13:28.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:28.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6e3fff640 10 monclient: dump: 2023-11-02T16:13:28.264 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.264 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.265 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.266 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.266 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 10 client.21541.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:28.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 10 monclient: _renew_subs 2023-11-02T16:13:28.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe6f9ffb640 4 set_mon_vals no callback set 2023-11-02T16:13:28.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc11e4c0 con 0x7fe6fc142250 2023-11-02T16:13:28.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.232+0000 7fe701d0a640 1 librados: init done 2023-11-02T16:13:28.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.233+0000 7fe701d0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:28.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.233+0000 7fe701d0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:28.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.233+0000 7fe701d0a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.233+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc066cc0 con 0x7fe6fc142250 2023-11-02T16:13:28.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.250+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe6f4005470 con 0x7fe6fc142250 2023-11-02T16:13:28.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.251+0000 7fe6e3fff640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 0x7fe6e806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.251+0000 7fe6fb7fe640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 0x7fe6e806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.251+0000 7fe6fb7fe640 10 monclient: get_auth_request con 0x7fe6e806be10 auth_method 0 2023-11-02T16:13:28.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.251+0000 7fe6e3fff640 10 client.21541.objecter ms_handle_connect 0x7fe6e806be10 2023-11-02T16:13:28.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.251+0000 7fe6fb7fe640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 0x7fe6e806e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0010e0 tx=0x7fe6ec007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.472 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.470+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fe6f40583b0 con 0x7fe6fc142250 2023-11-02T16:13:28.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.470+0000 7fe6e3fff640 10 client.21541.objecter ms_dispatch 0x7fe6fc000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:28.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.470+0000 7fe6e3fff640 3 client.21541.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:28.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.470+0000 7fe6e3fff640 3 client.21541.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:28.473 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.471+0000 7fe6e3fff640 20 client.21541.objecter dump_active .. 0 homeless 2023-11-02T16:13:28.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.481+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe6f40546f0 con 0x7fe6fc142250 2023-11-02T16:13:28.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.481+0000 7fe6e3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:28.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.481+0000 7fe6e3fff640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:28.653 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.651+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6fc0ae630 con 0x7fe6e806be10 2023-11-02T16:13:28.653 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.651+0000 7fe6e3fff640 1 -- 172.21.15.174:0/2407048871 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6ec007620 con 0x7fe6e806be10 2023-11-02T16:13:28.653 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 2 to deep-scrub 2023-11-02T16:13:28.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 10 librados: watch_flush enter 2023-11-02T16:13:28.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 10 librados: watch_flush exit 2023-11-02T16:13:28.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 20 client.21541.objecter shutdown clearing up homeless session... 2023-11-02T16:13:28.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 10 client.21541.objecter successfully canceled tick 2023-11-02T16:13:28.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 msgr2=0x7fe6e806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 0x7fe6e806e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0010e0 tx=0x7fe6ec007be0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.658 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 10 monclient: shutdown 2023-11-02T16:13:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 msgr2=0x7fe6fc11dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 secure :-1 s=READY pgs=5267 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c6e0 tx=0x7fe6f400cba0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 shutdown_connections 2023-11-02T16:13:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc142250 0x7fe6fc11dda0 unknown :-1 s=CLOSED pgs=5267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6fc0ae1c0 0x7fe6fc11d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fe6e806be10 0x7fe6e806e2d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 --2- 172.21.15.174:0/2407048871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6fc06b7d0 0x7fe6fc1da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 >> 172.21.15.174:0/2407048871 conn(0x7fe6fc0b5060 msgr2=0x7fe6fc0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:28.660 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 shutdown_connections 2023-11-02T16:13:28.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 -- 172.21.15.174:0/2407048871 wait complete. 2023-11-02T16:13:28.661 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.654+0000 7fe701d0a640 1 librados: shutdown 2023-11-02T16:13:28.668 INFO:tasks.ceph:Scrubbing osd.3 2023-11-02T16:13:28.668 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:28.775 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.773+0000 7f1609ac4640 10 monclient: get_monmap_and_config 2023-11-02T16:13:28.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.774+0000 7f1609ac4640 10 monclient: build_initial_monmap 2023-11-02T16:13:28.776 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.774+0000 7f1609ac4640 10 monclient: monmap: 2023-11-02T16:13:28.776 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:28.776 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.776 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:28.774921+0000 2023-11-02T16:13:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:28.774921+0000 2023-11-02T16:13:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:28.778 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:28.778 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:28.778 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:28.778 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.774+0000 7f1609ac4640 1 Processor -- start 2023-11-02T16:13:28.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.774+0000 7f1609ac4640 1 -- start start 2023-11-02T16:13:28.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.774+0000 7f1609ac4640 10 monclient: init 2023-11-02T16:13:28.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2023-11-02T16:13:28.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f16040b3c10 0x7f160413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-c con 0x7f16040b3c10 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:28.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f160413ede0 0x7f16040b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-a con 0x7f160413ede0 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:28.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f16040b0a00 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:28.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16040af970 con 0x7f16040b0a00 2023-11-02T16:13:28.782 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16040b3ff0 con 0x7f160413ede0 2023-11-02T16:13:28.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160413f250 con 0x7f16040b3c10 2023-11-02T16:13:28.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.783 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.775+0000 7f1609ac4640 10 monclient(hunting): authenticate will time out at 3736.117143s 2023-11-02T16:13:28.791 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.790+0000 7f16037fe640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f16040b3c10 0x7f160413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.790+0000 7f1602ffd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f160413ede0 0x7f16040b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1603fff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f16040b0a00 0x7f16040b1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51412/0 (socket says 172.21.15.174:51412) 2023-11-02T16:13:28.792 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1603fff640 1 -- 172.21.15.174:0/3854271030 learned_addr learned my addr 172.21.15.174:0/3854271030 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:28.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f16037fe640 10 monclient(hunting): get_auth_request con 0x7f16040b3c10 auth_method 0 2023-11-02T16:13:28.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f16037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f16037fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.793 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f16037fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1602ffd640 10 monclient(hunting): get_auth_request con 0x7f160413ede0 auth_method 0 2023-11-02T16:13:28.794 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1602ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1602ffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.791+0000 7f1602ffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.792+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.795 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.792+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.796 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.792+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 10 monclient(hunting): handle_auth_done global_id 21544 payload 274 2023-11-02T16:13:28.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 1 -- 172.21.15.174:0/3854271030 >> v2:172.21.15.174:3300/0 conn(0x7f160413ede0 msgr2=0x7f16040b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:28.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 1 --2- 172.21.15.174:0/3854271030 >> v2:172.21.15.174:3300/0 conn(0x7f160413ede0 0x7f16040b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 1 -- 172.21.15.174:0/3854271030 >> v1:172.21.15.175:6789/0 conn(0x7f16040b0a00 legacy=0x7f16040b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 1 monclient: found mon.noname-c 2023-11-02T16:13:28.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.795+0000 7f16037fe640 1 -- 172.21.15.174:0/3854271030 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16040afbc0 con 0x7f16040b3c10 2023-11-02T16:13:28.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.799+0000 7f16037fe640 1 --2- 172.21.15.174:0/3854271030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16040b3c10 0x7f160413e8a0 secure :-1 s=READY pgs=5269 cs=0 l=1 rev1=1 crypto rx=0x7f16040b4ac0 tx=0x7f15f402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b475b99468ad7b server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 1 -- 172.21.15.174:0/3854271030 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f402cbf0 con 0x7f16040b3c10 2023-11-02T16:13:28.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:28.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: dump: 2023-11-02T16:13:28.834 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.834 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.835 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.835 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.835 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.835 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.835 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.836 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.836 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.836 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.836 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 1 -- 172.21.15.174:0/3854271030 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f402cd50 con 0x7f16040b3c10 2023-11-02T16:13:28.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f1609ac4640 5 monclient: authenticate success, global_id 21544 2023-11-02T16:13:28.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f1609ac4640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:28.837 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f1609ac4640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:28.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f1609ac4640 4 set_mon_vals no callback set 2023-11-02T16:13:28.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 1 -- 172.21.15.174:0/3854271030 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4034660 con 0x7f16040b3c10 2023-11-02T16:13:28.838 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.831+0000 7f16027fc640 10 monclient: dump: 2023-11-02T16:13:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.840 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.840 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.840 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.840 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.840 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.841 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.841 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 10 monclient: shutdown 2023-11-02T16:13:28.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:28.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 -- 172.21.15.174:0/3854271030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16040b3c10 msgr2=0x7f160413e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 --2- 172.21.15.174:0/3854271030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16040b3c10 0x7f160413e8a0 secure :-1 s=READY pgs=5269 cs=0 l=1 rev1=1 crypto rx=0x7f16040b4ac0 tx=0x7f15f402be40 comp rx=0 tx=0).stop 2023-11-02T16:13:28.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 -- 172.21.15.174:0/3854271030 shutdown_connections 2023-11-02T16:13:28.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 --2- 172.21.15.174:0/3854271030 >> v2:172.21.15.174:3300/0 conn(0x7f160413ede0 0x7f16040b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 --2- 172.21.15.174:0/3854271030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16040b3c10 0x7f160413e8a0 unknown :-1 s=CLOSED pgs=5269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 -- 172.21.15.174:0/3854271030 >> 172.21.15.174:0/3854271030 conn(0x7f1604065960 msgr2=0x7f1604065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:28.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 -- 172.21.15.174:0/3854271030 shutdown_connections 2023-11-02T16:13:28.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.832+0000 7f1609ac4640 1 -- 172.21.15.174:0/3854271030 wait complete. 2023-11-02T16:13:28.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.833+0000 7f1609ac4640 10 monclient: build_initial_monmap 2023-11-02T16:13:28.847 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.833+0000 7f1609ac4640 10 monclient: monmap: 2023-11-02T16:13:28.847 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:28.848 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:28.848 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:28.848 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:28.848 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:28.848 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.849 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:28.849 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:28.849 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:28.849 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.849 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 librados: starting msgr at 2023-11-02T16:13:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 librados: starting objecter 2023-11-02T16:13:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 Processor -- start 2023-11-02T16:13:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 -- start start 2023-11-02T16:13:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 librados: setting wanted keys 2023-11-02T16:13:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 1 librados: calling monclient init 2023-11-02T16:13:28.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.834+0000 7f1609ac4640 10 monclient: init 2023-11-02T16:13:28.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:28.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T16:13:28.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16040b0a00 0x7f1604140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-c con 0x7f16040b0a00 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:28.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16040b3c10 0x7f160413a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-a con 0x7f16040b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:28.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160413ede0 0x7f16041d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): picked mon.noname-b con 0x7f160413ede0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:28.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16041d6930 con 0x7f16040b3c10 2023-11-02T16:13:28.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16041d6b80 con 0x7f160413ede0 2023-11-02T16:13:28.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:28.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16041411a0 con 0x7f16040b0a00 2023-11-02T16:13:28.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:28.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.835+0000 7f1609ac4640 10 monclient(hunting): authenticate will time out at 3736.177282s 2023-11-02T16:13:28.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16040b0a00 0x7f1604140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f1603fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160413ede0 0x7f16041d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16040b0a00 0x7f1604140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54672/0 (socket says 172.21.15.174:54672) 2023-11-02T16:13:28.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 1 -- 172.21.15.174:0/297370735 learned_addr learned my addr 172.21.15.174:0/297370735 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:28.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 10 monclient(hunting): get_auth_request con 0x7f16040b0a00 auth_method 0 2023-11-02T16:13:28.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f16037fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f1603fff640 10 monclient(hunting): get_auth_request con 0x7f160413ede0 auth_method 0 2023-11-02T16:13:28.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f1603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:28.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f1603fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:28.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.836+0000 7f1603fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:28.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f1603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f1603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f1603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:28.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:28.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.837+0000 7f16037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:28.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 10 monclient(hunting): handle_auth_done global_id 21550 payload 824 2023-11-02T16:13:28.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 -- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16040b0a00 msgr2=0x7f1604140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16040b0a00 0x7f1604140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 -- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16040b3c10 msgr2=0x7f160413a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:28.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16040b3c10 0x7f160413a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:28.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 10 monclient: _finish_hunting 0 2023-11-02T16:13:28.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 monclient: found mon.noname-b 2023-11-02T16:13:28.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:28.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.839+0000 7f1603fff640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009a70 con 0x7f160413ede0 2023-11-02T16:13:28.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.843+0000 7f1603fff640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160413ede0 0x7f16041d70a0 secure :-1 s=READY pgs=5270 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c3c0 tx=0x7f15fc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.843+0000 7f15fbfff640 10 client.?.objecter ms_handle_connect 0x7f160413ede0 2023-11-02T16:13:28.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.843+0000 7f15fbfff640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:28.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc005ce0 con 0x7f160413ede0 2023-11-02T16:13:28.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:28.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: dump: 2023-11-02T16:13:28.876 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.876 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.877 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.877 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.877 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.877 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.877 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.878 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.878 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.878 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: _finish_auth 0 2023-11-02T16:13:28.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: _check_auth_tickets 2023-11-02T16:13:28.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:28.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc005e40 con 0x7f160413ede0 2023-11-02T16:13:28.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 5 monclient: authenticate success, global_id 21550 2023-11-02T16:13:28.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 10 monclient: _renew_subs 2023-11-02T16:13:28.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1604141500 con 0x7f160413ede0 2023-11-02T16:13:28.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:28.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0072d0 con 0x7f160413ede0 2023-11-02T16:13:28.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f15fbfff640 10 monclient: dump: 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.881 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:28.882 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:28.882 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:28.882 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:28.882 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:28.882 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:28.883 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:28.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f16017fa640 4 set_mon_vals no callback set 2023-11-02T16:13:28.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 10 client.21550.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:28.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 10 monclient: _renew_subs 2023-11-02T16:13:28.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160413f660 con 0x7f160413ede0 2023-11-02T16:13:28.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.872+0000 7f1609ac4640 1 librados: init done 2023-11-02T16:13:28.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 10 client.21550.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:28.885 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 15 client.21550.objecter _session_command_op_assign -1 1 2023-11-02T16:13:28.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 20 client.21550.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:28.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 10 client.21550.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:28.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:28.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.873+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16040ae380 con 0x7f160413ede0 2023-11-02T16:13:28.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.942+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f15fc0074f0 con 0x7f160413ede0 2023-11-02T16:13:28.945 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.943+0000 7f15fbfff640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f15e406be10 0x7f15e406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.945 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.943+0000 7f16037fe640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f15e406be10 0x7f15e406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.946 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.943+0000 7f16037fe640 10 monclient: get_auth_request con 0x7f15e406be10 auth_method 0 2023-11-02T16:13:28.946 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.944+0000 7f16037fe640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f15e406be10 0x7f15e406e2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15f4000f80 tx=0x7f15f4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.946 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.944+0000 7f15fbfff640 10 client.21550.objecter ms_handle_connect 0x7f15e406be10 2023-11-02T16:13:28.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f15fc057850 con 0x7f160413ede0 2023-11-02T16:13:28.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 10 client.21550.objecter ms_dispatch 0x7f1604000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:28.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 3 client.21550.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 3 client.21550.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 10 client.21550.objecter checking command 1 2023-11-02T16:13:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f15e4073780 0x7f15e4075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter _get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 15 client.21550.objecter _session_command_op_remove -1 1 2023-11-02T16:13:28.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter _get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 15 client.21550.objecter _session_command_op_assign 3 1 2023-11-02T16:13:28.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 4 2023-11-02T16:13:28.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 10 client.21550.objecter _send_command 1 2023-11-02T16:13:28.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15e40762f0 con 0x7f15e4073780 2023-11-02T16:13:28.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.945+0000 7f15fbfff640 20 client.21550.objecter dump_active .. 0 homeless 2023-11-02T16:13:28.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.946+0000 7f1602ffd640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f15e4073780 0x7f15e4075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:28.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.946+0000 7f1602ffd640 10 monclient: get_auth_request con 0x7f15e4073780 auth_method 0 2023-11-02T16:13:28.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.950+0000 7f1602ffd640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f15e4073780 0x7f15e4075c20 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:28.952 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.950+0000 7f15fbfff640 10 client.21550.objecter ms_handle_connect 0x7f15e4073780 2023-11-02T16:13:28.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.973+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== osd.3 v2:172.21.15.174:6818/86148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f15e40762f0 con 0x7f15e4073780 2023-11-02T16:13:28.975 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.973+0000 7f15fbfff640 10 client.21550.objecter ms_dispatch 0x7f1604000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:28.976 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.973+0000 7f15fbfff640 10 client.21550.objecter _finish_command 1 = system:0 2023-11-02T16:13:28.976 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.973+0000 7f15fbfff640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 4 2023-11-02T16:13:28.976 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.973+0000 7f15fbfff640 15 client.21550.objecter _session_command_op_remove 3 1 2023-11-02T16:13:28.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 10 client.21550.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:28.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 15 client.21550.objecter _session_command_op_assign -1 2 2023-11-02T16:13:28.988 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 20 client.21550.objecter _get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 20 client.21550.objecter _get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 15 client.21550.objecter _session_command_op_remove -1 2 2023-11-02T16:13:28.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 20 client.21550.objecter get_session s=0x7f15e40730b0 osd=3 3 2023-11-02T16:13:28.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 15 client.21550.objecter _session_command_op_assign 3 2 2023-11-02T16:13:28.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 4 2023-11-02T16:13:28.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 10 client.21550.objecter _send_command 2 2023-11-02T16:13:28.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.986+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 --> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f16040afea0 con 0x7f15e4073780 2023-11-02T16:13:28.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.996+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15fc053b90 con 0x7f160413ede0 2023-11-02T16:13:28.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:28.996+0000 7f15fbfff640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:29.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f15fbfff640 1 -- 172.21.15.174:0/297370735 <== osd.3 v2:172.21.15.174:6818/86148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f15ec00b9a0 con 0x7f15e4073780 2023-11-02T16:13:29.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f15fbfff640 10 client.21550.objecter ms_dispatch 0x7f1604000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:29.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f15fbfff640 10 client.21550.objecter _finish_command 2 = system:0 2023-11-02T16:13:29.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f15fbfff640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 4 2023-11-02T16:13:29.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f15fbfff640 15 client.21550.objecter _session_command_op_remove 3 2 2023-11-02T16:13:29.007 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:29.007 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:29.007 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 10 librados: watch_flush enter 2023-11-02T16:13:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 10 librados: watch_flush exit 2023-11-02T16:13:29.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 10 client.21550.objecter close_session for osd.3 2023-11-02T16:13:29.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f15e4073780 msgr2=0x7f15e4075c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6818/86148,v1:172.21.15.174:6820/86148] conn(0x7f15e4073780 0x7f15e4075c20 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 20 client.21550.objecter put_session s=0x7f15e40730b0 osd=3 1 2023-11-02T16:13:29.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 20 client.21550.objecter shutdown clearing up homeless session... 2023-11-02T16:13:29.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 10 client.21550.objecter successfully canceled tick 2023-11-02T16:13:29.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f15e406be10 msgr2=0x7f15e406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f15e406be10 0x7f15e406e2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f15f4000f80 tx=0x7f15f4030000 comp rx=0 tx=0).stop 2023-11-02T16:13:29.010 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 10 monclient: shutdown 2023-11-02T16:13:29.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:29.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160413ede0 msgr2=0x7f16041d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 --2- 172.21.15.174:0/297370735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f160413ede0 0x7f16041d70a0 secure :-1 s=READY pgs=5270 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c3c0 tx=0x7f15fc00c880 comp rx=0 tx=0).stop 2023-11-02T16:13:29.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1603fff640 1 -- 172.21.15.174:0/297370735 reap_dead start 2023-11-02T16:13:29.011 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 shutdown_connections 2023-11-02T16:13:29.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 >> 172.21.15.174:0/297370735 conn(0x7f1604065960 msgr2=0x7f16040ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:29.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 shutdown_connections 2023-11-02T16:13:29.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 -- 172.21.15.174:0/297370735 wait complete. 2023-11-02T16:13:29.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.004+0000 7f1609ac4640 1 librados: shutdown 2023-11-02T16:13:29.017 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-02T16:13:29.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.146+0000 7fa0c159f640 10 monclient: get_monmap_and_config 2023-11-02T16:13:29.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.146+0000 7fa0c159f640 10 monclient: build_initial_monmap 2023-11-02T16:13:29.148 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.146+0000 7fa0c159f640 10 monclient: monmap: 2023-11-02T16:13:29.149 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:29.149 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.149 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:29.147372+0000 2023-11-02T16:13:29.149 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:29.147372+0000 2023-11-02T16:13:29.149 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:29.150 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.150 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:29.150 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:29.150 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:29.151 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:29.151 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:29.151 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 Processor -- start 2023-11-02T16:13:29.154 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 -- start start 2023-11-02T16:13:29.155 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient: init 2023-11-02T16:13:29.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:29.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2023-11-02T16:13:29.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-b con 0x7fa0bc0b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:29.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-a con 0x7fa0bc0b0ed0 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:29.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa0bc0b1cb0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:29.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc0aa1d0 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b4950 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0aa4b0 con 0x7fa0bc0b4570 2023-11-02T16:13:29.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:29.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.147+0000 7fa0c159f640 10 monclient(hunting): authenticate will time out at 3736.489337s 2023-11-02T16:13:29.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40900/0 (socket says 172.21.15.174:40900) 2023-11-02T16:13:29.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/772563575 learned_addr learned my addr 172.21.15.174:0/772563575 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:29.162 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 10 monclient(hunting): get_auth_request con 0x7fa0bc0b0ed0 auth_method 0 2023-11-02T16:13:29.163 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.163 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.148+0000 7fa0ba7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.149+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.164 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.149+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.149+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_done global_id 15822 payload 274 2023-11-02T16:13:29.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 msgr2=0x7fa0bc144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:29.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 1 --2- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/772563575 >> v1:172.21.15.175:6789/0 conn(0x7fa0bc0b1cb0 legacy=0x7fa0bc0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:29.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 10 monclient: _finish_hunting 0 2023-11-02T16:13:29.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 1 monclient: found mon.noname-a 2023-11-02T16:13:29.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:29.169 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:29.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.150+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/772563575 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0a8b50 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.170 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.152+0000 7fa0ba7fc640 1 --2- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009850 tx=0x7fa0b002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2f81df73319cce server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:29.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 1 -- 172.21.15.174:0/772563575 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0004230 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:29.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: dump: 2023-11-02T16:13:29.179 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.179 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.180 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.180 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.180 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.181 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.181 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.181 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.181 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.181 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: _finish_auth 0 2023-11-02T16:13:29.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: _check_auth_tickets 2023-11-02T16:13:29.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:29.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 1 -- 172.21.15.174:0/772563575 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b002f050 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:29.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 1 -- 172.21.15.174:0/772563575 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003610 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 5 monclient: authenticate success, global_id 15822 2023-11-02T16:13:29.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:29.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 4 set_mon_vals no callback set 2023-11-02T16:13:29.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:29.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0b9ffb640 10 monclient: dump: 2023-11-02T16:13:29.188 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.188 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.188 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.189 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.189 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.189 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.189 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.189 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.190 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.190 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 10 monclient: shutdown 2023-11-02T16:13:29.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:29.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 -- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 msgr2=0x7fa0bc0a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 --2- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009850 tx=0x7fa0b002b7b0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 -- 172.21.15.174:0/772563575 shutdown_connections 2023-11-02T16:13:29.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 --2- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0a9070 unknown :-1 s=CLOSED pgs=3193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 --2- 172.21.15.174:0/772563575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 -- 172.21.15.174:0/772563575 >> 172.21.15.174:0/772563575 conn(0x7fa0bc065d50 msgr2=0x7fa0bc066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:29.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 -- 172.21.15.174:0/772563575 shutdown_connections 2023-11-02T16:13:29.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.157+0000 7fa0c159f640 1 -- 172.21.15.174:0/772563575 wait complete. 2023-11-02T16:13:29.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.158+0000 7fa0c159f640 10 monclient: build_initial_monmap 2023-11-02T16:13:29.194 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.158+0000 7fa0c159f640 10 monclient: monmap: 2023-11-02T16:13:29.194 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:29.195 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:29.195 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:29.196 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:29.196 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:29.197 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.197 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:29.197 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:29.198 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:29.198 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.158+0000 7fa0c159f640 1 librados: starting msgr at 2023-11-02T16:13:29.198 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.158+0000 7fa0c159f640 1 librados: starting objecter 2023-11-02T16:13:29.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 Processor -- start 2023-11-02T16:13:29.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 -- start start 2023-11-02T16:13:29.199 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 librados: setting wanted keys 2023-11-02T16:13:29.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 librados: calling monclient init 2023-11-02T16:13:29.200 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient: init 2023-11-02T16:13:29.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:29.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T16:13:29.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.201 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-a con 0x7fa0bc0b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:29.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-b con 0x7fa0bc0b1cb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:29.202 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): picked mon.noname-c con 0x7fa0bc0b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0ab640 con 0x7fa0bc0b0ed0 2023-11-02T16:13:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc13fb20 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc140580 con 0x7fa0bc0b4570 2023-11-02T16:13:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:29.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.159+0000 7fa0c159f640 10 monclient(hunting): authenticate will time out at 3736.501552s 2023-11-02T16:13:29.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54876/0 (socket says 172.21.15.174:54876) 2023-11-02T16:13:29.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/437171449 learned_addr learned my addr 172.21.15.174:0/437171449 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:29.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 10 monclient(hunting): get_auth_request con 0x7fa0bc0b1cb0 auth_method 0 2023-11-02T16:13:29.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.160+0000 7fa0ba7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): get_auth_request con 0x7fa0bc0b4570 auth_method 0 2023-11-02T16:13:29.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.161+0000 7fa0bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.163+0000 7fa0baffd640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 10 monclient(hunting): handle_auth_done global_id 21553 payload 824 2023-11-02T16:13:29.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 msgr2=0x7fa0bc140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.211 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 msgr2=0x7fa0bc0b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 10 monclient: _finish_hunting 0 2023-11-02T16:13:29.212 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 monclient: found mon.noname-b 2023-11-02T16:13:29.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:29.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:29.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.164+0000 7fa0ba7fc640 1 -- 172.21.15.174:0/437171449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0009460 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.213 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.166+0000 7fa0ba7fc640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 secure :-1 s=READY pgs=5271 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0036040 tx=0x7fa0b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:29.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.166+0000 7fa0a37fe640 10 client.?.objecter ms_handle_connect 0x7fa0bc0b1cb0 2023-11-02T16:13:29.214 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.166+0000 7fa0a37fe640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:29.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003b60 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:29.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: dump: 2023-11-02T16:13:29.218 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.218 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.218 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.218 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.219 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.219 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.219 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.219 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.219 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.220 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:29.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:29.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:29.220 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003cc0 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 5 monclient: authenticate success, global_id 21553 2023-11-02T16:13:29.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 10 monclient: _renew_subs 2023-11-02T16:13:29.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:29.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc13f640 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:29.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0006260 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:29.223 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0a37fe640 10 monclient: dump: 2023-11-02T16:13:29.223 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.223 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.223 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.224 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.224 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.224 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.225 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.225 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.225 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0b97fa640 4 set_mon_vals no callback set 2023-11-02T16:13:29.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 10 client.21553.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:29.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 10 monclient: _renew_subs 2023-11-02T16:13:29.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:29.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc0aff90 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.194+0000 7fa0c159f640 1 librados: init done 2023-11-02T16:13:29.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.195+0000 7fa0c159f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:29.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.195+0000 7fa0c159f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:29.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.195+0000 7fa0c159f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:29.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.195+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc13f940 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.213+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0b003a040 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.213+0000 7fa0a37fe640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 0x7fa0a806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.213+0000 7fa0baffd640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 0x7fa0a806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.213+0000 7fa0baffd640 10 monclient: get_auth_request con 0x7fa0a806be10 auth_method 0 2023-11-02T16:13:29.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.214+0000 7fa0a37fe640 10 client.21553.objecter ms_handle_connect 0x7fa0a806be10 2023-11-02T16:13:29.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.214+0000 7fa0baffd640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 0x7fa0a806e2d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a770 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:29.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.216+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fa0b0003e30 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.216+0000 7fa0a37fe640 10 client.21553.objecter ms_dispatch 0x7fa0bc000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:29.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.216+0000 7fa0a37fe640 3 client.21553.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:29.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.216+0000 7fa0a37fe640 3 client.21553.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:29.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.216+0000 7fa0a37fe640 20 client.21553.objecter dump_active .. 0 homeless 2023-11-02T16:13:29.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.473+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa0b0075ca0 con 0x7fa0bc0b1cb0 2023-11-02T16:13:29.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.473+0000 7fa0a37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:29.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.473+0000 7fa0a37fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:29.598 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:41.752 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.622 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:41.776 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.650 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:41.803 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.652 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:41.805 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.666 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:41.762 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.671 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:41.768 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.678 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:41.832 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.685 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.684+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0bc0ab7b0 con 0x7fa0a806be10 2023-11-02T16:13:29.686 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.684+0000 7fa0a37fe640 1 -- 172.21.15.174:0/437171449 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa0bc0ab7b0 con 0x7fa0a806be10 2023-11-02T16:13:29.686 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 3 to deep-scrub 2023-11-02T16:13:29.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 10 librados: watch_flush enter 2023-11-02T16:13:29.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 10 librados: watch_flush exit 2023-11-02T16:13:29.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 20 client.21553.objecter shutdown clearing up homeless session... 2023-11-02T16:13:29.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 10 client.21553.objecter successfully canceled tick 2023-11-02T16:13:29.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 msgr2=0x7fa0a806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 0x7fa0a806e2d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a770 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2023-11-02T16:13:29.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 10 monclient: shutdown 2023-11-02T16:13:29.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:29.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 msgr2=0x7fa0bc141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 secure :-1 s=READY pgs=5271 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0036040 tx=0x7fa0b00046c0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.690 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 shutdown_connections 2023-11-02T16:13:29.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa0a806be10 0x7fa0a806e2d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0bc0b4570 0x7fa0bc140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.691 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0bc0b1cb0 0x7fa0bc141250 unknown :-1 s=CLOSED pgs=5271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 --2- 172.21.15.174:0/437171449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0bc0b0ed0 0x7fa0bc0b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 >> 172.21.15.174:0/437171449 conn(0x7fa0bc065d50 msgr2=0x7fa0bc13ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:29.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 shutdown_connections 2023-11-02T16:13:29.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 -- 172.21.15.174:0/437171449 wait complete. 2023-11-02T16:13:29.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.686+0000 7fa0c159f640 1 librados: shutdown 2023-11-02T16:13:29.698 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:41.794 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.701 INFO:tasks.ceph:Scrubbing osd.4 2023-11-02T16:13:29.701 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:29.711 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:41.864 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:29.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.806+0000 7f618038b640 10 monclient: get_monmap_and_config 2023-11-02T16:13:29.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.806+0000 7f618038b640 10 monclient: build_initial_monmap 2023-11-02T16:13:29.808 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.806+0000 7f618038b640 10 monclient: monmap: 2023-11-02T16:13:29.809 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:29.809 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.809 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:29.807356+0000 2023-11-02T16:13:29.809 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:29.807356+0000 2023-11-02T16:13:29.809 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:29.810 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.810 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:29.810 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:29.810 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:29.811 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:29.811 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:29.811 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 1 Processor -- start 2023-11-02T16:13:29.812 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 1 -- start start 2023-11-02T16:13:29.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 10 monclient: init 2023-11-02T16:13:29.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:29.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2023-11-02T16:13:29.813 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f61780b3c10 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:29.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.807+0000 7f618038b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 0x7f61780b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): picked mon.noname-b con 0x7f617813efb0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:29.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 0x7f617814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): picked mon.noname-c con 0x7f61780b0a00 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:29.814 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780af970 con 0x7f61780b3c10 2023-11-02T16:13:29.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780b3ff0 con 0x7f61780b0a00 2023-11-02T16:13:29.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780afbc0 con 0x7f617813efb0 2023-11-02T16:13:29.815 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:29.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.808+0000 7f618038b640 10 monclient(hunting): authenticate will time out at 3737.149805s 2023-11-02T16:13:29.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.812+0000 7f617e928640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 0x7f617814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.816 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.813+0000 7f617d926640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 0x7f61780b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 0x7f617814bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54880/0 (socket says 172.21.15.174:54880) 2023-11-02T16:13:29.818 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 1 -- 172.21.15.174:0/2148503907 learned_addr learned my addr 172.21.15.174:0/2148503907 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:29.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 10 monclient(hunting): get_auth_request con 0x7f61780b0a00 auth_method 0 2023-11-02T16:13:29.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.819 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.816+0000 7f617e928640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.819+0000 7f617e928640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.819+0000 7f617e928640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.819+0000 7f617e928640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.862+0000 7f617d926640 10 monclient(hunting): get_auth_request con 0x7f617813efb0 auth_method 0 2023-11-02T16:13:29.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.862+0000 7f617d926640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.862+0000 7f617d926640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.864 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.862+0000 7f617d926640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.867 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.866+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.867 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.866+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.866+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.871 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 10 monclient(hunting): handle_auth_done global_id 16946 payload 274 2023-11-02T16:13:29.871 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 1 -- 172.21.15.174:0/2148503907 >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 msgr2=0x7f617814bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:29.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 1 --2- 172.21.15.174:0/2148503907 >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 0x7f617814bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 1 -- 172.21.15.174:0/2148503907 >> v1:172.21.15.174:6789/0 conn(0x7f61780b3c10 legacy=0x7f617813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 10 monclient: _finish_hunting 0 2023-11-02T16:13:29.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 1 monclient: found mon.noname-b 2023-11-02T16:13:29.872 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:29.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:29.873 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.870+0000 7f617d926640 1 -- 172.21.15.174:0/2148503907 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780afea0 con 0x7f617813efb0 2023-11-02T16:13:29.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.886+0000 7f617d926640 1 --2- 172.21.15.174:0/2148503907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 0x7f61780b04c0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f61680097a0 tx=0x7f616802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511ae81cb68f0787 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:29.928 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 1 -- 172.21.15.174:0/2148503907 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616802e070 con 0x7f617813efb0 2023-11-02T16:13:29.929 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.929 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:29.929 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: dump: 2023-11-02T16:13:29.929 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.930 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.930 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.930 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.930 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.931 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.931 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.931 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.931 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.933 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.933 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: _finish_auth 0 2023-11-02T16:13:29.933 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: _check_auth_tickets 2023-11-02T16:13:29.933 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:29.934 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 1 -- 172.21.15.174:0/2148503907 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61680315a0 con 0x7f617813efb0 2023-11-02T16:13:29.934 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f618038b640 5 monclient: authenticate success, global_id 16946 2023-11-02T16:13:29.934 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f618038b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:29.934 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:29.934 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f618038b640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:29.935 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f618038b640 4 set_mon_vals no callback set 2023-11-02T16:13:29.935 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 1 -- 172.21.15.174:0/2148503907 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168031930 con 0x7f617813efb0 2023-11-02T16:13:29.936 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.936 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:29.936 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.926+0000 7f617d125640 10 monclient: dump: 2023-11-02T16:13:29.936 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.936 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.937 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.937 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.937 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.937 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.938 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.938 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.938 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.939 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.941 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 10 monclient: shutdown 2023-11-02T16:13:29.941 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:29.942 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 -- 172.21.15.174:0/2148503907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 msgr2=0x7f61780b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.942 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 --2- 172.21.15.174:0/2148503907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 0x7f61780b04c0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f61680097a0 tx=0x7f616802b840 comp rx=0 tx=0).stop 2023-11-02T16:13:29.942 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 -- 172.21.15.174:0/2148503907 shutdown_connections 2023-11-02T16:13:29.943 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 --2- 172.21.15.174:0/2148503907 >> v2:172.21.15.175:3300/0 conn(0x7f61780b0a00 0x7f617814bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.943 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 --2- 172.21.15.174:0/2148503907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f617813efb0 0x7f61780b04c0 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.943 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 -- 172.21.15.174:0/2148503907 >> 172.21.15.174:0/2148503907 conn(0x7f6178065960 msgr2=0x7f6178065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:29.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 -- 172.21.15.174:0/2148503907 shutdown_connections 2023-11-02T16:13:29.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.927+0000 7f618038b640 1 -- 172.21.15.174:0/2148503907 wait complete. 2023-11-02T16:13:29.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 10 monclient: build_initial_monmap 2023-11-02T16:13:29.944 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 10 monclient: monmap: 2023-11-02T16:13:29.945 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:29.945 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:29.945 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:29.945 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:29.945 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:29.946 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.946 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:29.946 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:29.946 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:29.946 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 librados: starting msgr at 2023-11-02T16:13:29.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 librados: starting objecter 2023-11-02T16:13:29.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 Processor -- start 2023-11-02T16:13:29.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 -- start start 2023-11-02T16:13:29.947 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 librados: setting wanted keys 2023-11-02T16:13:29.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 1 librados: calling monclient init 2023-11-02T16:13:29.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.928+0000 7f618038b640 10 monclient: init 2023-11-02T16:13:29.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:29.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T16:13:29.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 0x7f61780a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): picked mon.noname-c con 0x7f61780b0a00 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:29.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61780b3c10 0x7f61780aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): picked mon.noname-a con 0x7f61780b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:29.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f617813efb0 0x7f61780ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:29.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): picked mon.noname-b con 0x7f617813efb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780ab7e0 con 0x7f61780b3c10 2023-11-02T16:13:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780a8580 con 0x7f617813efb0 2023-11-02T16:13:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:29.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780ad190 con 0x7f61780b0a00 2023-11-02T16:13:29.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:29.951 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.929+0000 7f618038b640 10 monclient(hunting): authenticate will time out at 3737.271162s 2023-11-02T16:13:29.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.943+0000 7f617e127640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 0x7f61780a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617d926640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61780b3c10 0x7f61780aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 0x7f61780a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54706/0 (socket says 172.21.15.174:54706) 2023-11-02T16:13:29.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 1 -- 172.21.15.174:0/804902772 learned_addr learned my addr 172.21.15.174:0/804902772 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:29.955 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 10 monclient(hunting): get_auth_request con 0x7f61780b0a00 auth_method 0 2023-11-02T16:13:29.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617e127640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617d926640 10 monclient(hunting): get_auth_request con 0x7f61780b3c10 auth_method 0 2023-11-02T16:13:29.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617d926640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:29.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617d926640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:29.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.945+0000 7f617d926640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:29.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.946+0000 7f617e127640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.946+0000 7f617e127640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.957 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.946+0000 7f617e127640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.947+0000 7f617e928640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f617813efb0 0x7f61780ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:29.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.950+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:29.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.950+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:29.958 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.950+0000 7f617d926640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:29.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 10 monclient(hunting): handle_auth_done global_id 16949 payload 824 2023-11-02T16:13:29.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 -- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f617813efb0 msgr2=0x7f61780ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f617813efb0 0x7f61780ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 -- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61780b3c10 msgr2=0x7f61780aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:29.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61780b3c10 0x7f61780aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:29.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 10 monclient: _finish_hunting 0 2023-11-02T16:13:29.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 monclient: found mon.noname-c 2023-11-02T16:13:29.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:29.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:29.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.952+0000 7f617e127640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009400 con 0x7f61780b0a00 2023-11-02T16:13:29.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.961+0000 7f617e127640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 0x7f61780a9e90 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f617400c450 tx=0x7f617400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:29.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.961+0000 7f616effd640 10 client.?.objecter ms_handle_connect 0x7f61780b0a00 2023-11-02T16:13:29.963 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.961+0000 7f616effd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:29.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61740052d0 con 0x7f61780b0a00 2023-11-02T16:13:29.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:29.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: dump: 2023-11-02T16:13:29.995 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:29.995 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:29.995 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.995 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:29.995 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:29.996 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:29.996 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:29.996 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:29.996 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:29.996 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:29.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: _finish_auth 0 2023-11-02T16:13:29.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:29.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:29.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174005430 con 0x7f61780b0a00 2023-11-02T16:13:29.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 5 monclient: authenticate success, global_id 16949 2023-11-02T16:13:29.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 10 monclient: _renew_subs 2023-11-02T16:13:29.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:29.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61780ac5e0 con 0x7f61780b0a00 2023-11-02T16:13:29.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:29.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174012870 con 0x7f61780b0a00 2023-11-02T16:13:29.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:29.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:30.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f616effd640 10 monclient: dump: 2023-11-02T16:13:30.000 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:30.000 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.000 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.000 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.001 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:30.001 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.001 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:30.001 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:30.001 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:30.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f617c924640 4 set_mon_vals no callback set 2023-11-02T16:13:30.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 10 client.16949.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:30.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 10 monclient: _renew_subs 2023-11-02T16:13:30.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:30.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61780aca60 con 0x7f61780b0a00 2023-11-02T16:13:30.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.991+0000 7f618038b640 1 librados: init done 2023-11-02T16:13:30.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 10 client.16949.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:30.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 15 client.16949.objecter _session_command_op_assign -1 1 2023-11-02T16:13:30.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 20 client.16949.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:30.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 10 client.16949.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:30.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:30.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:29.992+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61780ae010 con 0x7f61780b0a00 2023-11-02T16:13:30.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.026+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f61740129d0 con 0x7f61780b0a00 2023-11-02T16:13:30.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.026+0000 7f616effd640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f615c06be10 0x7f615c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.027+0000 7f617d926640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f615c06be10 0x7f615c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.027+0000 7f617d926640 10 monclient: get_auth_request con 0x7f615c06be10 auth_method 0 2023-11-02T16:13:30.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.027+0000 7f616effd640 10 client.16949.objecter ms_handle_connect 0x7f615c06be10 2023-11-02T16:13:30.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.027+0000 7f617d926640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f615c06be10 0x7f615c06e2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6168036000 tx=0x7f616803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:30.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.030+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f61740587a0 con 0x7f61780b0a00 2023-11-02T16:13:30.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.030+0000 7f616effd640 10 client.16949.objecter ms_dispatch 0x7f6178000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:30.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.030+0000 7f616effd640 3 client.16949.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:30.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.030+0000 7f616effd640 3 client.16949.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:30.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 10 client.16949.objecter checking command 1 2023-11-02T16:13:30.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7f615c073780 0x7f615c075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter _get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.034 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.035 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 15 client.16949.objecter _session_command_op_remove -1 1 2023-11-02T16:13:30.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter _get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 15 client.16949.objecter _session_command_op_assign 4 1 2023-11-02T16:13:30.037 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 4 2023-11-02T16:13:30.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 10 client.16949.objecter _send_command 1 2023-11-02T16:13:30.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f615c0762f0 con 0x7f615c073780 2023-11-02T16:13:30.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 20 client.16949.objecter dump_active .. 0 homeless 2023-11-02T16:13:30.038 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61740011e0 con 0x7f61780b0a00 2023-11-02T16:13:30.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f616effd640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:30.039 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.031+0000 7f617e928640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7f615c073780 0x7f615c075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.032+0000 7f617e928640 10 monclient: get_auth_request con 0x7f615c073780 auth_method 0 2023-11-02T16:13:30.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.035+0000 7f616effd640 10 client.16949.objecter ms_handle_connect 0x7f615c073780 2023-11-02T16:13:30.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.035+0000 7f617e928640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7f615c073780 0x7f615c075c20 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:30.059 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.058+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== osd.4 v2:172.21.15.175:6816/86415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f615c0762f0 con 0x7f615c073780 2023-11-02T16:13:30.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.058+0000 7f616effd640 10 client.16949.objecter ms_dispatch 0x7f6178000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:30.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.058+0000 7f616effd640 10 client.16949.objecter _finish_command 1 = system:0 2023-11-02T16:13:30.060 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.058+0000 7f616effd640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 4 2023-11-02T16:13:30.061 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.058+0000 7f616effd640 15 client.16949.objecter _session_command_op_remove 4 1 2023-11-02T16:13:30.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 10 client.16949.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:30.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 15 client.16949.objecter _session_command_op_assign -1 2 2023-11-02T16:13:30.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 20 client.16949.objecter _get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 20 client.16949.objecter _get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 15 client.16949.objecter _session_command_op_remove -1 2 2023-11-02T16:13:30.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 20 client.16949.objecter get_session s=0x7f615c0730b0 osd=4 3 2023-11-02T16:13:30.075 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 15 client.16949.objecter _session_command_op_assign 4 2 2023-11-02T16:13:30.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 4 2023-11-02T16:13:30.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 10 client.16949.objecter _send_command 2 2023-11-02T16:13:30.076 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.071+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 --> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f61780ae380 con 0x7f615c073780 2023-11-02T16:13:30.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f616effd640 1 -- 172.21.15.174:0/804902772 <== osd.4 v2:172.21.15.175:6816/86415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6170009ca0 con 0x7f615c073780 2023-11-02T16:13:30.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f616effd640 10 client.16949.objecter ms_dispatch 0x7f6178000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:30.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f616effd640 10 client.16949.objecter _finish_command 2 = system:0 2023-11-02T16:13:30.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f616effd640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 4 2023-11-02T16:13:30.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f616effd640 15 client.16949.objecter _session_command_op_remove 4 2 2023-11-02T16:13:30.090 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:30.091 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:30.091 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:30.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 10 librados: watch_flush enter 2023-11-02T16:13:30.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 10 librados: watch_flush exit 2023-11-02T16:13:30.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 10 client.16949.objecter close_session for osd.4 2023-11-02T16:13:30.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7f615c073780 msgr2=0x7f615c075c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.175:6816/86415,v1:172.21.15.175:6817/86415] conn(0x7f615c073780 0x7f615c075c20 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 20 client.16949.objecter put_session s=0x7f615c0730b0 osd=4 1 2023-11-02T16:13:30.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 20 client.16949.objecter shutdown clearing up homeless session... 2023-11-02T16:13:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 10 client.16949.objecter successfully canceled tick 2023-11-02T16:13:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f615c06be10 msgr2=0x7f615c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.086+0000 7f618038b640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f615c06be10 0x7f615c06e2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6168036000 tx=0x7f616803d040 comp rx=0 tx=0).stop 2023-11-02T16:13:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 10 monclient: shutdown 2023-11-02T16:13:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 msgr2=0x7f61780a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 --2- 172.21.15.174:0/804902772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61780b0a00 0x7f61780a9e90 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f617400c450 tx=0x7f617400c910 comp rx=0 tx=0).stop 2023-11-02T16:13:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f617e928640 1 -- 172.21.15.174:0/804902772 reap_dead start 2023-11-02T16:13:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 shutdown_connections 2023-11-02T16:13:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 >> 172.21.15.174:0/804902772 conn(0x7f6178065960 msgr2=0x7f61780b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 shutdown_connections 2023-11-02T16:13:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 -- 172.21.15.174:0/804902772 wait complete. 2023-11-02T16:13:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.087+0000 7f618038b640 1 librados: shutdown 2023-11-02T16:13:30.100 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-02T16:13:30.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 10 monclient: get_monmap_and_config 2023-11-02T16:13:30.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 10 monclient: build_initial_monmap 2023-11-02T16:13:30.244 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 10 monclient: monmap: 2023-11-02T16:13:30.245 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:30.245 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.245 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:30.243079+0000 2023-11-02T16:13:30.245 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:30.243079+0000 2023-11-02T16:13:30.245 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:30.246 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.246 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:30.246 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:30.246 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:30.247 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:30.247 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:30.247 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 1 Processor -- start 2023-11-02T16:13:30.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 1 -- start start 2023-11-02T16:13:30.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.242+0000 7f1cb87c9640 10 monclient: init 2023-11-02T16:13:30.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:30.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient: _add_conns ranks=[4,1,2,0,3] 2023-11-02T16:13:30.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1cb00b4570 0x7f1cb0144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-c con 0x7f1cb00b4570 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:30.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1cb00b0ed0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:30.250 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb013e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-b con 0x7f1cb013e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:30.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00a9780 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:30.251 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00b4950 con 0x7f1cb00b4570 2023-11-02T16:13:30.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:30.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00aa1d0 con 0x7f1cb013e520 2023-11-02T16:13:30.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:30.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.243+0000 7f1cb87c9640 10 monclient(hunting): authenticate will time out at 3737.585274s 2023-11-02T16:13:30.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.248+0000 7f1cb6565640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1cb00b4570 0x7f1cb0144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.251+0000 7f1cb5d64640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f1cb00b0ed0 0x7f1cb00a9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51426/0 (socket says 172.21.15.174:51426) 2023-11-02T16:13:30.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.251+0000 7f1cb5d64640 1 -- 172.21.15.174:0/2714338555 learned_addr learned my addr 172.21.15.174:0/2714338555 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:30.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.253+0000 7f1cb6565640 10 monclient(hunting): get_auth_request con 0x7f1cb00b4570 auth_method 0 2023-11-02T16:13:30.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.253+0000 7f1cb6565640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:30.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.253+0000 7f1cb6565640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:30.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.253+0000 7f1cb6565640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:30.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.282+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:30.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.282+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:30.284 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.282+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:30.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.293+0000 7f1cb6565640 10 monclient(hunting): handle_auth_done global_id 21559 payload 274 2023-11-02T16:13:30.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.293+0000 7f1cb6565640 1 -- 172.21.15.174:0/2714338555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 msgr2=0x7f1cb013e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:30.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.293+0000 7f1cb6565640 1 --2- 172.21.15.174:0/2714338555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb013e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 1 -- 172.21.15.174:0/2714338555 >> v1:172.21.15.175:6789/0 conn(0x7f1cb00b0ed0 legacy=0x7f1cb00a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 10 monclient: _finish_hunting 0 2023-11-02T16:13:30.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 1 monclient: found mon.noname-c 2023-11-02T16:13:30.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:30.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:30.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.294+0000 7f1cb6565640 1 -- 172.21.15.174:0/2714338555 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00aa4b0 con 0x7f1cb00b4570 2023-11-02T16:13:30.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.301+0000 7f1cb6565640 1 --2- 172.21.15.174:0/2714338555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb0144d40 secure :-1 s=READY pgs=5273 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097d0 tx=0x7f1cac02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ebdd1bd054b157 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:30.316 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 1 -- 172.21.15.174:0/2714338555 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0316f0 con 0x7f1cb00b4570 2023-11-02T16:13:30.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:30.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:30.317 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: dump: 2023-11-02T16:13:30.317 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:30.318 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.318 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.318 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.318 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:30.319 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.319 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:30.319 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:30.319 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:30.319 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: _finish_auth 0 2023-11-02T16:13:30.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: _check_auth_tickets 2023-11-02T16:13:30.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:30.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 1 -- 172.21.15.174:0/2714338555 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac031850 con 0x7f1cb00b4570 2023-11-02T16:13:30.320 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:30.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 1 -- 172.21.15.174:0/2714338555 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac031be0 con 0x7f1cb00b4570 2023-11-02T16:13:30.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb87c9640 5 monclient: authenticate success, global_id 21559 2023-11-02T16:13:30.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb87c9640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:30.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:30.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb87c9640 4 set_mon_vals no callback set 2023-11-02T16:13:30.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:30.322 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.315+0000 7f1cb5563640 10 monclient: dump: 2023-11-02T16:13:30.322 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:30.322 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.323 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.323 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.323 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:30.323 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.323 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:30.324 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:30.325 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:30.325 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 10 monclient: shutdown 2023-11-02T16:13:30.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:30.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2714338555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 msgr2=0x7f1cb0144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2714338555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb0144d40 secure :-1 s=READY pgs=5273 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097d0 tx=0x7f1cac02b790 comp rx=0 tx=0).stop 2023-11-02T16:13:30.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2714338555 shutdown_connections 2023-11-02T16:13:30.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2714338555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb013e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2714338555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb0144d40 unknown :-1 s=CLOSED pgs=5273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2714338555 >> 172.21.15.174:0/2714338555 conn(0x7f1cb0065d50 msgr2=0x7f1cb0066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:30.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2714338555 shutdown_connections 2023-11-02T16:13:30.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.316+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2714338555 wait complete. 2023-11-02T16:13:30.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 10 monclient: build_initial_monmap 2023-11-02T16:13:30.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 10 monclient: monmap: 2023-11-02T16:13:30.331 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:30.331 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:30.331 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:30.331 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:30.331 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:30.332 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.332 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:30.332 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:30.333 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:30.333 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 librados: starting msgr at 2023-11-02T16:13:30.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 librados: starting objecter 2023-11-02T16:13:30.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 Processor -- start 2023-11-02T16:13:30.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 -- start start 2023-11-02T16:13:30.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 librados: setting wanted keys 2023-11-02T16:13:30.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 1 librados: calling monclient init 2023-11-02T16:13:30.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.317+0000 7f1cb87c9640 10 monclient: init 2023-11-02T16:13:30.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:30.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T16:13:30.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.336 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-a con 0x7f1cb00b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:30.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb01d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-b con 0x7f1cb00b4570 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:30.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb00af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): picked mon.noname-c con 0x7f1cb013e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:30.337 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:30.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00aeca0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:30.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00ae080 con 0x7f1cb00b4570 2023-11-02T16:13:30.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:30.338 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb01da530 con 0x7f1cb013e520 2023-11-02T16:13:30.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:30.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.318+0000 7f1cb87c9640 10 monclient(hunting): authenticate will time out at 3737.660480s 2023-11-02T16:13:30.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.322+0000 7f1cb6565640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40922/0 (socket says 172.21.15.174:40922) 2023-11-02T16:13:30.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 1 -- 172.21.15.174:0/2282981461 learned_addr learned my addr 172.21.15.174:0/2282981461 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:30.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 10 monclient(hunting): get_auth_request con 0x7f1cb00b0ed0 auth_method 0 2023-11-02T16:13:30.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:30.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:30.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.323+0000 7f1cb6565640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:30.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.325+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:30.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.325+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:30.341 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.325+0000 7f1cb6565640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:30.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.330+0000 7f1cb5d64640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb01d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.332+0000 7f1cb5d64640 10 monclient(hunting): get_auth_request con 0x7f1cb00b4570 auth_method 0 2023-11-02T16:13:30.343 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.332+0000 7f1cb5d64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:30.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.332+0000 7f1cb5d64640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:30.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.332+0000 7f1cb5d64640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:30.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 10 monclient(hunting): handle_auth_done global_id 15828 payload 824 2023-11-02T16:13:30.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 -- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 msgr2=0x7f1cb00af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.344 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb00af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 -- 172.21.15.174:0/2282981461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 msgr2=0x7f1cb01d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb01d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 10 monclient: _finish_hunting 0 2023-11-02T16:13:30.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 monclient: found mon.noname-a 2023-11-02T16:13:30.345 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:30.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:30.346 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.333+0000 7f1cb6565640 1 -- 172.21.15.174:0/2282981461 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac009400 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.356+0000 7f1cb6565640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0098f0 tx=0x7f1cac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:30.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.356+0000 7f1ca6ffd640 10 client.?.objecter ms_handle_connect 0x7f1cb00b0ed0 2023-11-02T16:13:30.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.356+0000 7f1ca6ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:30.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac034890 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: dump: 2023-11-02T16:13:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:30.488 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.488 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.488 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.488 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:30.488 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:30.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:30.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:30.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac034a10 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 5 monclient: authenticate success, global_id 15828 2023-11-02T16:13:30.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 10 monclient: _renew_subs 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00af6e0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac034dc0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:30.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:30.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1ca6ffd640 10 monclient: dump: 2023-11-02T16:13:30.492 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:30.492 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:30.492 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.492 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:30.493 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:30.493 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:30.493 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:30.493 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:30.493 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:30.494 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:30.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb4d62640 4 set_mon_vals no callback set 2023-11-02T16:13:30.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 10 client.15828.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:30.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 10 monclient: _renew_subs 2023-11-02T16:13:30.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb00afbf0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.484+0000 7f1cb87c9640 1 librados: init done 2023-11-02T16:13:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.485+0000 7f1cb87c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:30.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.485+0000 7f1cb87c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:30.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.485+0000 7f1cb87c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:30.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.485+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb00afea0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.645 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:42.741 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.646 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:42.742 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.655 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:42.751 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.759+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1cac03b040 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.760+0000 7f1ca6ffd640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 0x7f1c9406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:30.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.760+0000 7f1cb5d64640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 0x7f1c9406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:30.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.760+0000 7f1cb5d64640 10 monclient: get_auth_request con 0x7f1c9406be40 auth_method 0 2023-11-02T16:13:30.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.760+0000 7f1ca6ffd640 10 client.15828.objecter ms_handle_connect 0x7f1c9406be40 2023-11-02T16:13:30.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.760+0000 7f1cb5d64640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 0x7f1c9406e300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0470 tx=0x7f1ca0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:30.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.776+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f1cac033e90 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.776+0000 7f1ca6ffd640 10 client.15828.objecter ms_dispatch 0x7f1cb0000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:30.779 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.776+0000 7f1ca6ffd640 3 client.15828.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:30.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.776+0000 7f1ca6ffd640 3 client.15828.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:30.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.777+0000 7f1ca6ffd640 20 client.15828.objecter dump_active .. 0 homeless 2023-11-02T16:13:30.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.777+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1cac0761c0 con 0x7f1cb00b0ed0 2023-11-02T16:13:30.780 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.777+0000 7f1ca6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:30.781 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.777+0000 7f1ca6ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:30.826 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:42.980 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.828 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:42.981 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.838 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:42.991 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:30.973 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.971+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb00a8bb0 con 0x7f1c9406be40 2023-11-02T16:13:30.973 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.972+0000 7f1ca6ffd640 1 -- 172.21.15.174:0/2282981461 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1cb00a8bb0 con 0x7f1c9406be40 2023-11-02T16:13:30.974 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 4 to deep-scrub 2023-11-02T16:13:30.976 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 10 librados: watch_flush enter 2023-11-02T16:13:30.977 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 10 librados: watch_flush exit 2023-11-02T16:13:30.977 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 20 client.15828.objecter shutdown clearing up homeless session... 2023-11-02T16:13:30.977 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 10 client.15828.objecter successfully canceled tick 2023-11-02T16:13:30.977 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 msgr2=0x7f1c9406e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.978 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 0x7f1c9406e300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0470 tx=0x7f1ca0009040 comp rx=0 tx=0).stop 2023-11-02T16:13:30.978 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 10 monclient: shutdown 2023-11-02T16:13:30.978 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:30.978 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 msgr2=0x7f1cb01d9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:30.978 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0098f0 tx=0x7f1cac0046c0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 shutdown_connections 2023-11-02T16:13:30.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f1c9406be40 0x7f1c9406e300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cb013e520 0x7f1cb00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cb00b4570 0x7f1cb01d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.979 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 --2- 172.21.15.174:0/2282981461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cb00b0ed0 0x7f1cb01d9ab0 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:30.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 >> 172.21.15.174:0/2282981461 conn(0x7f1cb0065d50 msgr2=0x7f1cb00b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:30.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 shutdown_connections 2023-11-02T16:13:30.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 -- 172.21.15.174:0/2282981461 wait complete. 2023-11-02T16:13:30.980 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:30.974+0000 7f1cb87c9640 1 librados: shutdown 2023-11-02T16:13:30.987 INFO:tasks.ceph:Scrubbing osd.5 2023-11-02T16:13:30.988 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:31.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.121+0000 7f059e4b6640 10 monclient: get_monmap_and_config 2023-11-02T16:13:31.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.121+0000 7f059e4b6640 10 monclient: build_initial_monmap 2023-11-02T16:13:31.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.121+0000 7f059e4b6640 10 monclient: monmap: 2023-11-02T16:13:31.124 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:31.124 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.125 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:31.122319+0000 2023-11-02T16:13:31.125 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:31.122319+0000 2023-11-02T16:13:31.125 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:31.126 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.126 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:31.126 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:31.127 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:31.127 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:31.128 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:31.128 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 Processor -- start 2023-11-02T16:13:31.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 -- start start 2023-11-02T16:13:31.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient: init 2023-11-02T16:13:31.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:31.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2023-11-02T16:13:31.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f059813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-b con 0x7f05980b3c10 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:31.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f059813ede0 0x7f05980b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-c con 0x7f059813ede0 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:31.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f05980b0a00 0x7f05980b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-a con 0x7f05980b0a00 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:31.135 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980b3ff0 con 0x7f05980b0a00 2023-11-02T16:13:31.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980af970 con 0x7f059813ede0 2023-11-02T16:13:31.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980afbc0 con 0x7f05980b3c10 2023-11-02T16:13:31.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:31.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.122+0000 7f059e4b6640 10 monclient(hunting): authenticate will time out at 3738.464544s 2023-11-02T16:13:31.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.127+0000 7f059ca53640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f05980b0a00 0x7f05980b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f05980b0a00 0x7f05980b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40932/0 (socket says 172.21.15.174:40932) 2023-11-02T16:13:31.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 1 -- 172.21.15.174:0/1132180670 learned_addr learned my addr 172.21.15.174:0/1132180670 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:31.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 10 monclient(hunting): get_auth_request con 0x7f05980b0a00 auth_method 0 2023-11-02T16:13:31.140 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.140 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.128+0000 7f059ca53640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.130+0000 7f059ca53640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:31.141 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.130+0000 7f059ca53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:31.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.130+0000 7f059ca53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:31.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.130+0000 7f0597fff640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f059813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 10 monclient(hunting): handle_auth_done global_id 15831 payload 274 2023-11-02T16:13:31.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 -- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 msgr2=0x7f059813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:31.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f059813e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 -- 172.21.15.174:0/1132180670 >> v2:172.21.15.175:3300/0 conn(0x7f059813ede0 msgr2=0x7f05980b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:31.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 --2- 172.21.15.174:0/1132180670 >> v2:172.21.15.175:3300/0 conn(0x7f059813ede0 0x7f05980b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 10 monclient: _finish_hunting 0 2023-11-02T16:13:31.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 monclient: found mon.noname-a 2023-11-02T16:13:31.146 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:31.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:31.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.132+0000 7f059ca53640 1 -- 172.21.15.174:0/1132180670 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980afea0 con 0x7f05980b0a00 2023-11-02T16:13:31.147 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.135+0000 7f059ca53640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05980b0a00 0x7f05980b1950 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7f05980b4280 tx=0x7f059002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9133ee6b8c2dcbf server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 1 -- 172.21.15.174:0/1132180670 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059002c9e0 con 0x7f05980b0a00 2023-11-02T16:13:31.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:31.153 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: dump: 2023-11-02T16:13:31.153 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.154 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.154 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.154 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.155 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.155 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.156 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.156 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.156 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.157 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:31.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:31.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:31.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 1 -- 172.21.15.174:0/1132180670 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059002cb40 con 0x7f05980b0a00 2023-11-02T16:13:31.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:31.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 1 -- 172.21.15.174:0/1132180670 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900343f0 con 0x7f05980b0a00 2023-11-02T16:13:31.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f059e4b6640 5 monclient: authenticate success, global_id 15831 2023-11-02T16:13:31.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f059e4b6640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:31.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f059e4b6640 4 set_mon_vals no callback set 2023-11-02T16:13:31.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:31.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.147+0000 7f0596ffd640 10 monclient: dump: 2023-11-02T16:13:31.161 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.162 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.162 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.162 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.163 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.163 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.163 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.164 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.164 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.164 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 10 monclient: shutdown 2023-11-02T16:13:31.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:31.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 -- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05980b0a00 msgr2=0x7f05980b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05980b0a00 0x7f05980b1950 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7f05980b4280 tx=0x7f059002be40 comp rx=0 tx=0).stop 2023-11-02T16:13:31.165 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 -- 172.21.15.174:0/1132180670 shutdown_connections 2023-11-02T16:13:31.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05980b0a00 0x7f05980b1950 unknown :-1 s=CLOSED pgs=3198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 --2- 172.21.15.174:0/1132180670 >> v2:172.21.15.175:3300/0 conn(0x7f059813ede0 0x7f05980b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 --2- 172.21.15.174:0/1132180670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f059813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.166 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 -- 172.21.15.174:0/1132180670 >> 172.21.15.174:0/1132180670 conn(0x7f0598065960 msgr2=0x7f0598065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:31.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 -- 172.21.15.174:0/1132180670 shutdown_connections 2023-11-02T16:13:31.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 1 -- 172.21.15.174:0/1132180670 wait complete. 2023-11-02T16:13:31.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 10 monclient: build_initial_monmap 2023-11-02T16:13:31.167 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.148+0000 7f059e4b6640 10 monclient: monmap: 2023-11-02T16:13:31.167 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:31.168 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:31.168 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:31.168 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:31.169 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:31.169 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.169 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:31.170 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:31.170 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:31.170 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.171 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 librados: starting msgr at 2023-11-02T16:13:31.171 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 librados: starting objecter 2023-11-02T16:13:31.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 Processor -- start 2023-11-02T16:13:31.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 -- start start 2023-11-02T16:13:31.172 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 librados: setting wanted keys 2023-11-02T16:13:31.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 librados: calling monclient init 2023-11-02T16:13:31.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 10 monclient: init 2023-11-02T16:13:31.173 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:31.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:13:31.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05980b0a00 0x7f0598144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.174 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-b con 0x7f05980b0a00 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:31.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.149+0000 7f059e4b6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f05981d6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-c con 0x7f05980b3c10 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:31.175 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f059813ede0 0x7f05981d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): picked mon.noname-a con 0x7f059813ede0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:31.176 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059813f8b0 con 0x7f059813ede0 2023-11-02T16:13:31.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.177 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059813f480 con 0x7f05980b0a00 2023-11-02T16:13:31.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05981d7290 con 0x7f05980b3c10 2023-11-02T16:13:31.178 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:31.179 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.150+0000 7f059e4b6640 10 monclient(hunting): authenticate will time out at 3738.491703s 2023-11-02T16:13:31.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.154+0000 7f0597fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05980b0a00 0x7f0598144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.181 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.154+0000 7f05977fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f05981d6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05980b0a00 0x7f0598144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54912/0 (socket says 172.21.15.174:54912) 2023-11-02T16:13:31.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 1 -- 172.21.15.174:0/2968435235 learned_addr learned my addr 172.21.15.174:0/2968435235 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:31.182 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 10 monclient(hunting): get_auth_request con 0x7f05980b0a00 auth_method 0 2023-11-02T16:13:31.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.183 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.156+0000 7f0597fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.159+0000 7f0597fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:31.184 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.159+0000 7f0597fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:31.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.159+0000 7f0597fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:31.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.159+0000 7f059ca53640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f059813ede0 0x7f05981d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.185 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.161+0000 7f05977fe640 10 monclient(hunting): get_auth_request con 0x7f05980b3c10 auth_method 0 2023-11-02T16:13:31.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.161+0000 7f05977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.161+0000 7f05977fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.186 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.161+0000 7f05977fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.164+0000 7f05977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:31.187 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.164+0000 7f05977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:31.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.164+0000 7f05977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:31.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.168+0000 7f059ca53640 10 monclient(hunting): get_auth_request con 0x7f059813ede0 auth_method 0 2023-11-02T16:13:31.188 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.168+0000 7f059ca53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.168+0000 7f059ca53640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.168+0000 7f059ca53640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.189 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 10 monclient(hunting): handle_auth_done global_id 16952 payload 824 2023-11-02T16:13:31.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 -- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05980b0a00 msgr2=0x7f0598144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05980b0a00 0x7f0598144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.190 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 -- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f059813ede0 msgr2=0x7f05981d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.191 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f059813ede0 0x7f05981d6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:31.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 monclient: found mon.noname-c 2023-11-02T16:13:31.192 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:31.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:31.193 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.171+0000 7f05977fe640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009b30 con 0x7f05980b3c10 2023-11-02T16:13:31.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.205+0000 7f05977fe640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f05981d6810 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b440 tx=0x7f058c00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.205+0000 7f0594ff9640 10 client.?.objecter ms_handle_connect 0x7f05980b3c10 2023-11-02T16:13:31.210 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.205+0000 7f0594ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:31.221 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011830 con 0x7f05980b3c10 2023-11-02T16:13:31.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:31.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: dump: 2023-11-02T16:13:31.222 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.223 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.223 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.223 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.223 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.223 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: _finish_auth 0 2023-11-02T16:13:31.224 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: _check_auth_tickets 2023-11-02T16:13:31.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:31.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c011990 con 0x7f05980b3c10 2023-11-02T16:13:31.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:31.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011d40 con 0x7f05980b3c10 2023-11-02T16:13:31.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 5 monclient: authenticate success, global_id 16952 2023-11-02T16:13:31.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 10 monclient: _renew_subs 2023-11-02T16:13:31.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:31.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059813bbe0 con 0x7f05980b3c10 2023-11-02T16:13:31.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:31.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f05967fc640 4 set_mon_vals no callback set 2023-11-02T16:13:31.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f0594ff9640 10 monclient: dump: 2023-11-02T16:13:31.227 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.227 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.227 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.228 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.229 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 10 client.16952.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:31.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 10 monclient: _renew_subs 2023-11-02T16:13:31.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:31.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059813b290 con 0x7f05980b3c10 2023-11-02T16:13:31.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.219+0000 7f059e4b6640 1 librados: init done 2023-11-02T16:13:31.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 10 client.16952.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:31.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 15 client.16952.objecter _session_command_op_assign -1 1 2023-11-02T16:13:31.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 20 client.16952.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:31.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 10 client.16952.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:31.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:31.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.220+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f059813f220 con 0x7f05980b3c10 2023-11-02T16:13:31.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.316+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f058c01a400 con 0x7f05980b3c10 2023-11-02T16:13:31.318 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.316+0000 7f0594ff9640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f058406be10 0x7f058406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.316+0000 7f0597fff640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f058406be10 0x7f058406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.316+0000 7f0597fff640 10 monclient: get_auth_request con 0x7f058406be10 auth_method 0 2023-11-02T16:13:31.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.317+0000 7f0594ff9640 10 client.16952.objecter ms_handle_connect 0x7f058406be10 2023-11-02T16:13:31.319 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.317+0000 7f0597fff640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f058406be10 0x7f058406e2d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f058c058900 con 0x7f05980b3c10 2023-11-02T16:13:31.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 10 client.16952.objecter ms_dispatch 0x7f0598000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:31.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 3 client.16952.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:31.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 3 client.16952.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:31.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 10 client.16952.objecter checking command 1 2023-11-02T16:13:31.325 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f0584073780 0x7f0584075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.326 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter _get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 15 client.16952.objecter _session_command_op_remove -1 1 2023-11-02T16:13:31.327 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter _get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 15 client.16952.objecter _session_command_op_assign 5 1 2023-11-02T16:13:31.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 4 2023-11-02T16:13:31.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 10 client.16952.objecter _send_command 1 2023-11-02T16:13:31.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05840762f0 con 0x7f0584073780 2023-11-02T16:13:31.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 20 client.16952.objecter dump_active .. 0 homeless 2023-11-02T16:13:31.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058c011ea0 con 0x7f05980b3c10 2023-11-02T16:13:31.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.322+0000 7f0594ff9640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:31.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.323+0000 7f059ca53640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f0584073780 0x7f0584075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.323+0000 7f059ca53640 10 monclient: get_auth_request con 0x7f0584073780 auth_method 0 2023-11-02T16:13:31.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.327+0000 7f0594ff9640 10 client.16952.objecter ms_handle_connect 0x7f0584073780 2023-11-02T16:13:31.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.327+0000 7f059ca53640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f0584073780 0x7f0584075c20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.351+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== osd.5 v2:172.21.15.175:6808/86416 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f05840762f0 con 0x7f0584073780 2023-11-02T16:13:31.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.351+0000 7f0594ff9640 10 client.16952.objecter ms_dispatch 0x7f0598000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:31.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.351+0000 7f0594ff9640 10 client.16952.objecter _finish_command 1 = system:0 2023-11-02T16:13:31.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.351+0000 7f0594ff9640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 4 2023-11-02T16:13:31.353 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.351+0000 7f0594ff9640 15 client.16952.objecter _session_command_op_remove 5 1 2023-11-02T16:13:31.374 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 10 client.16952.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:31.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 15 client.16952.objecter _session_command_op_assign -1 2 2023-11-02T16:13:31.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 20 client.16952.objecter _get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.375 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 20 client.16952.objecter _get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 15 client.16952.objecter _session_command_op_remove -1 2 2023-11-02T16:13:31.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 20 client.16952.objecter get_session s=0x7f05840730b0 osd=5 3 2023-11-02T16:13:31.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 15 client.16952.objecter _session_command_op_assign 5 2 2023-11-02T16:13:31.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 4 2023-11-02T16:13:31.376 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 10 client.16952.objecter _send_command 2 2023-11-02T16:13:31.377 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.373+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 --> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f05980ae380 con 0x7f0584073780 2023-11-02T16:13:31.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f0594ff9640 1 -- 172.21.15.174:0/2968435235 <== osd.5 v2:172.21.15.175:6808/86416 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f059003e400 con 0x7f0584073780 2023-11-02T16:13:31.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f0594ff9640 10 client.16952.objecter ms_dispatch 0x7f0598000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:31.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f0594ff9640 10 client.16952.objecter _finish_command 2 = system:0 2023-11-02T16:13:31.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f0594ff9640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 4 2023-11-02T16:13:31.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f0594ff9640 15 client.16952.objecter _session_command_op_remove 5 2 2023-11-02T16:13:31.391 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:31.391 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:31.391 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:31.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 10 librados: watch_flush enter 2023-11-02T16:13:31.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 10 librados: watch_flush exit 2023-11-02T16:13:31.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 10 client.16952.objecter close_session for osd.5 2023-11-02T16:13:31.393 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f0584073780 msgr2=0x7f0584075c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.175:6808/86416,v1:172.21.15.175:6809/86416] conn(0x7f0584073780 0x7f0584075c20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 20 client.16952.objecter put_session s=0x7f05840730b0 osd=5 1 2023-11-02T16:13:31.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 20 client.16952.objecter shutdown clearing up homeless session... 2023-11-02T16:13:31.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 10 client.16952.objecter successfully canceled tick 2023-11-02T16:13:31.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f058406be10 msgr2=0x7f058406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f058406be10 0x7f058406e2d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588008040 comp rx=0 tx=0).stop 2023-11-02T16:13:31.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 10 monclient: shutdown 2023-11-02T16:13:31.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:31.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 msgr2=0x7f05981d6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 --2- 172.21.15.174:0/2968435235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05980b3c10 0x7f05981d6810 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b440 tx=0x7f058c00b900 comp rx=0 tx=0).stop 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059ca53640 1 -- 172.21.15.174:0/2968435235 reap_dead start 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 shutdown_connections 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.388+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 >> 172.21.15.174:0/2968435235 conn(0x7f0598065960 msgr2=0x7f0598073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.389+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 shutdown_connections 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.389+0000 7f059e4b6640 1 -- 172.21.15.174:0/2968435235 wait complete. 2023-11-02T16:13:31.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.389+0000 7f059e4b6640 1 librados: shutdown 2023-11-02T16:13:31.403 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-02T16:13:31.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.525+0000 7f82c5410640 10 monclient: get_monmap_and_config 2023-11-02T16:13:31.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.525+0000 7f82c5410640 10 monclient: build_initial_monmap 2023-11-02T16:13:31.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.525+0000 7f82c5410640 10 monclient: monmap: 2023-11-02T16:13:31.528 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:31.528 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.528 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:31.526783+0000 2023-11-02T16:13:31.529 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:31.526783+0000 2023-11-02T16:13:31.529 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:31.529 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.529 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:31.529 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:31.530 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:31.530 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:31.530 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:31.530 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.526+0000 7f82c5410640 1 Processor -- start 2023-11-02T16:13:31.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.526+0000 7f82c5410640 1 -- start start 2023-11-02T16:13:31.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.526+0000 7f82c5410640 10 monclient: init 2023-11-02T16:13:31.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:31.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2023-11-02T16:13:31.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f82c006aa00 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:31.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f82c00adfa0 0x7f82c013fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-c con 0x7f82c00adfa0 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:31.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c0149f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-b con 0x7f82c01400e0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:31.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c0143b30 con 0x7f82c006aa00 2023-11-02T16:13:31.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c00af200 con 0x7f82c00adfa0 2023-11-02T16:13:31.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c006ade0 con 0x7f82c01400e0 2023-11-02T16:13:31.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:31.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.527+0000 7f82c5410640 10 monclient(hunting): authenticate will time out at 3738.868842s 2023-11-02T16:13:31.544 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.542+0000 7f82be7fc640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f82c00adfa0 0x7f82c013fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82beffd640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f82c006aa00 0x7f82c013ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51430/0 (socket says 172.21.15.174:51430) 2023-11-02T16:13:31.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82beffd640 1 -- 172.21.15.174:0/4031230968 learned_addr learned my addr 172.21.15.174:0/4031230968 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:31.546 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82be7fc640 10 monclient(hunting): get_auth_request con 0x7f82c00adfa0 auth_method 0 2023-11-02T16:13:31.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82be7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.547 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.544+0000 7f82be7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.545+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:31.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.545+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:31.548 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.545+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:31.551 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 10 monclient(hunting): handle_auth_done global_id 21568 payload 274 2023-11-02T16:13:31.551 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 1 -- 172.21.15.174:0/4031230968 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 msgr2=0x7f82c0149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:31.551 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 1 --2- 172.21.15.174:0/4031230968 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c0149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 1 -- 172.21.15.174:0/4031230968 >> v1:172.21.15.175:6789/0 conn(0x7f82c006aa00 legacy=0x7f82c013ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 10 monclient: _finish_hunting 0 2023-11-02T16:13:31.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 1 monclient: found mon.noname-c 2023-11-02T16:13:31.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:31.552 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:31.553 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.549+0000 7f82be7fc640 1 -- 172.21.15.174:0/4031230968 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00ae410 con 0x7f82c00adfa0 2023-11-02T16:13:31.563 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.562+0000 7f82be7fc640 1 --2- 172.21.15.174:0/4031230968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c00adfa0 0x7f82c013fba0 secure :-1 s=READY pgs=5275 cs=0 l=1 rev1=1 crypto rx=0x7f82b40097d0 tx=0x7f82b402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2193a3ff41998a73 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.586 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 1 -- 172.21.15.174:0/4031230968 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b40316f0 con 0x7f82c00adfa0 2023-11-02T16:13:31.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:31.587 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: dump: 2023-11-02T16:13:31.587 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.588 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.588 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.588 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.588 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.588 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.589 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.589 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.589 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.589 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.589 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: _finish_auth 0 2023-11-02T16:13:31.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: _check_auth_tickets 2023-11-02T16:13:31.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:31.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 1 -- 172.21.15.174:0/4031230968 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4031850 con 0x7f82c00adfa0 2023-11-02T16:13:31.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:31.590 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 5 monclient: authenticate success, global_id 21568 2023-11-02T16:13:31.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 1 -- 172.21.15.174:0/4031230968 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4031be0 con 0x7f82c00adfa0 2023-11-02T16:13:31.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:31.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 4 set_mon_vals no callback set 2023-11-02T16:13:31.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.591 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:31.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82bdffb640 10 monclient: dump: 2023-11-02T16:13:31.592 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.592 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.592 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.592 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.593 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.593 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.593 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.593 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.593 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.594 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 10 monclient: shutdown 2023-11-02T16:13:31.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:31.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 -- 172.21.15.174:0/4031230968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c00adfa0 msgr2=0x7f82c013fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.594 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 --2- 172.21.15.174:0/4031230968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c00adfa0 0x7f82c013fba0 secure :-1 s=READY pgs=5275 cs=0 l=1 rev1=1 crypto rx=0x7f82b40097d0 tx=0x7f82b402b790 comp rx=0 tx=0).stop 2023-11-02T16:13:31.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 -- 172.21.15.174:0/4031230968 shutdown_connections 2023-11-02T16:13:31.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 --2- 172.21.15.174:0/4031230968 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c0149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 --2- 172.21.15.174:0/4031230968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c00adfa0 0x7f82c013fba0 unknown :-1 s=CLOSED pgs=5275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.582+0000 7f82c5410640 1 -- 172.21.15.174:0/4031230968 >> 172.21.15.174:0/4031230968 conn(0x7f82c0069c70 msgr2=0x7f82c006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:31.595 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 1 -- 172.21.15.174:0/4031230968 shutdown_connections 2023-11-02T16:13:31.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 1 -- 172.21.15.174:0/4031230968 wait complete. 2023-11-02T16:13:31.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 10 monclient: build_initial_monmap 2023-11-02T16:13:31.596 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 10 monclient: monmap: 2023-11-02T16:13:31.596 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:31.596 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:31.597 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:31.597 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:31.597 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:31.597 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.597 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:31.598 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:31.598 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:31.598 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.598 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 1 librados: starting msgr at 2023-11-02T16:13:31.598 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.583+0000 7f82c5410640 1 librados: starting objecter 2023-11-02T16:13:31.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 Processor -- start 2023-11-02T16:13:31.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 -- start start 2023-11-02T16:13:31.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 librados: setting wanted keys 2023-11-02T16:13:31.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 librados: calling monclient init 2023-11-02T16:13:31.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient: init 2023-11-02T16:13:31.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:31.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient: _add_conns ranks=[1,0,2] 2023-11-02T16:13:31.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c006aa00 0x7f82c01391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-b con 0x7f82c006aa00 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:31.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-a con 0x7f82c00adfa0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:31.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c01d6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): picked mon.noname-c con 0x7f82c01400e0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:31.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.601 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01d5c90 con 0x7f82c00adfa0 2023-11-02T16:13:31.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01d5ee0 con 0x7f82c006aa00 2023-11-02T16:13:31.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:31.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0139fa0 con 0x7f82c01400e0 2023-11-02T16:13:31.602 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:31.603 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.584+0000 7f82c5410640 10 monclient(hunting): authenticate will time out at 3738.926499s 2023-11-02T16:13:31.604 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.587+0000 7f82be7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.604 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40946/0 (socket says 172.21.15.174:40946) 2023-11-02T16:13:31.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 1 -- 172.21.15.174:0/705556893 learned_addr learned my addr 172.21.15.174:0/705556893 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:31.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 10 monclient(hunting): get_auth_request con 0x7f82c00adfa0 auth_method 0 2023-11-02T16:13:31.605 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:31.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:31.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.588+0000 7f82be7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:31.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.589+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:31.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.589+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:31.606 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.589+0000 7f82be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:31.607 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.592+0000 7f82be7fc640 10 monclient(hunting): handle_auth_done global_id 15837 payload 824 2023-11-02T16:13:31.607 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.592+0000 7f82be7fc640 1 -- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 msgr2=0x7f82c01d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.609 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c01d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 1 -- 172.21.15.174:0/705556893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c006aa00 msgr2=0x7f82c01391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:31.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c006aa00 0x7f82c01391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:31.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 10 monclient: _finish_hunting 0 2023-11-02T16:13:31.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 1 monclient: found mon.noname-a 2023-11-02T16:13:31.610 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:31.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:31.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.593+0000 7f82be7fc640 1 -- 172.21.15.174:0/705556893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009400 con 0x7f82c00adfa0 2023-11-02T16:13:31.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.601+0000 7f82be7fc640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c00 tx=0x7f82b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.601+0000 7f82a77fe640 10 client.?.objecter ms_handle_connect 0x7f82c00adfa0 2023-11-02T16:13:31.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.601+0000 7f82a77fe640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:31.641 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4034890 con 0x7f82c00adfa0 2023-11-02T16:13:31.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:31.642 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 10 monclient: dump: 2023-11-02T16:13:31.642 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.642 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.643 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.643 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.643 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.643 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:31.644 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:31.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:31.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.638+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4031f00 con 0x7f82c00adfa0 2023-11-02T16:13:31.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 5 monclient: authenticate success, global_id 15837 2023-11-02T16:13:31.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 10 monclient: _renew_subs 2023-11-02T16:13:31.645 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:31.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c01396f0 con 0x7f82c00adfa0 2023-11-02T16:13:31.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82a77fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:31.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4034c30 con 0x7f82c00adfa0 2023-11-02T16:13:31.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:31.646 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:31.647 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82a77fe640 10 monclient: dump: 2023-11-02T16:13:31.647 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:31.647 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:31.647 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.648 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:31.648 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:31.648 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:31.648 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:31.648 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:31.649 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:31.649 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:31.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82bd7fa640 4 set_mon_vals no callback set 2023-11-02T16:13:31.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 10 client.15837.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:31.649 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 10 monclient: _renew_subs 2023-11-02T16:13:31.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:31.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c01d6940 con 0x7f82c00adfa0 2023-11-02T16:13:31.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.639+0000 7f82c5410640 1 librados: init done 2023-11-02T16:13:31.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.640+0000 7f82c5410640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:31.650 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.640+0000 7f82c5410640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:31.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.640+0000 7f82c5410640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:31.651 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.640+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c01d6b30 con 0x7f82c00adfa0 2023-11-02T16:13:31.897 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.895+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f82b403b040 con 0x7f82c00adfa0 2023-11-02T16:13:31.897 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.895+0000 7f82a77fe640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 0x7f82ac06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:31.898 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.896+0000 7f82beffd640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 0x7f82ac06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:31.898 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.896+0000 7f82beffd640 10 monclient: get_auth_request con 0x7f82ac06be10 auth_method 0 2023-11-02T16:13:31.898 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.896+0000 7f82a77fe640 10 client.15837.objecter ms_handle_connect 0x7f82ac06be10 2023-11-02T16:13:31.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.896+0000 7f82beffd640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 0x7f82ac06e2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f82b0001550 tx=0x7f82b0007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:31.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f82b407a460 con 0x7f82c00adfa0 2023-11-02T16:13:31.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 10 client.15837.objecter ms_dispatch 0x7f82c0000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:31.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 3 client.15837.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 3 client.15837.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 20 client.15837.objecter dump_active .. 0 homeless 2023-11-02T16:13:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f82b40760a0 con 0x7f82c00adfa0 2023-11-02T16:13:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:31.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:31.911+0000 7f82a77fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:31.921 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:44.074 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:31.933 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:44.087 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:31.996 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:44.149 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:32.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.091+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f82c0139880 con 0x7f82ac06be10 2023-11-02T16:13:32.094 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.092+0000 7f82a77fe640 1 -- 172.21.15.174:0/705556893 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f82c0139880 con 0x7f82ac06be10 2023-11-02T16:13:32.094 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 5 to deep-scrub 2023-11-02T16:13:32.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.093+0000 7f82c5410640 10 librados: watch_flush enter 2023-11-02T16:13:32.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.093+0000 7f82c5410640 10 librados: watch_flush exit 2023-11-02T16:13:32.096 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.093+0000 7f82c5410640 20 client.15837.objecter shutdown clearing up homeless session... 2023-11-02T16:13:32.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.093+0000 7f82c5410640 10 client.15837.objecter successfully canceled tick 2023-11-02T16:13:32.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 msgr2=0x7f82ac06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 0x7f82ac06e2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f82b0001550 tx=0x7f82b0007a40 comp rx=0 tx=0).stop 2023-11-02T16:13:32.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 10 monclient: shutdown 2023-11-02T16:13:32.097 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:32.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 msgr2=0x7f82c0142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c00 tx=0x7f82b40046c0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 shutdown_connections 2023-11-02T16:13:32.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f82ac06be10 0x7f82ac06e2d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.098 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82c01400e0 0x7f82c01d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82c00adfa0 0x7f82c0142330 unknown :-1 s=CLOSED pgs=3199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 --2- 172.21.15.174:0/705556893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82c006aa00 0x7f82c01391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 >> 172.21.15.174:0/705556893 conn(0x7f82c0069c70 msgr2=0x7f82c00a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:32.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 shutdown_connections 2023-11-02T16:13:32.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 -- 172.21.15.174:0/705556893 wait complete. 2023-11-02T16:13:32.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.094+0000 7f82c5410640 1 librados: shutdown 2023-11-02T16:13:32.106 INFO:tasks.ceph:Scrubbing osd.6 2023-11-02T16:13:32.107 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:32.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.216+0000 7efeca71b640 10 monclient: get_monmap_and_config 2023-11-02T16:13:32.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.216+0000 7efeca71b640 10 monclient: build_initial_monmap 2023-11-02T16:13:32.219 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.217+0000 7efeca71b640 10 monclient: monmap: 2023-11-02T16:13:32.219 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:32.219 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.220 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:32.217886+0000 2023-11-02T16:13:32.220 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:32.217886+0000 2023-11-02T16:13:32.220 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:32.220 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.221 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:32.221 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:32.221 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:32.221 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:32.221 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:32.222 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.217+0000 7efeca71b640 1 Processor -- start 2023-11-02T16:13:32.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.217+0000 7efeca71b640 1 -- start start 2023-11-02T16:13:32.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.217+0000 7efeca71b640 10 monclient: init 2023-11-02T16:13:32.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:32.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2023-11-02T16:13:32.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efec40b3c10 0x7efec413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-c con 0x7efec40b3c10 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:32.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efec413ede0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:32.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efec40b0bd0 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:32.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec40af970 con 0x7efec40b0bd0 2023-11-02T16:13:32.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec413f250 con 0x7efec413ede0 2023-11-02T16:13:32.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec40b3ff0 con 0x7efec40b3c10 2023-11-02T16:13:32.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:32.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.218+0000 7efeca71b640 10 monclient(hunting): authenticate will time out at 3739.559985s 2023-11-02T16:13:32.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.221+0000 7efec8cb8640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7efec40b0bd0 0x7efec414bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:39860/0 (socket says 172.21.15.174:39860) 2023-11-02T16:13:32.230 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.221+0000 7efec8cb8640 1 -- 172.21.15.174:0/976100372 learned_addr learned my addr 172.21.15.174:0/976100372 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:32.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.223+0000 7efec3fff640 1 --2- 172.21.15.174:0/976100372 >> v2:172.21.15.175:3300/0 conn(0x7efec40b3c10 0x7efec413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.224+0000 7efec3fff640 10 monclient(hunting): get_auth_request con 0x7efec40b3c10 auth_method 0 2023-11-02T16:13:32.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.224+0000 7efec3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:32.231 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.224+0000 7efec3fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:32.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.224+0000 7efec3fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:32.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.225+0000 7efec3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:32.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.225+0000 7efec3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:32.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.225+0000 7efec3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:32.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 10 monclient(hunting): handle_auth_done global_id 21574 payload 274 2023-11-02T16:13:32.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 1 -- 172.21.15.174:0/976100372 >> v1:172.21.15.175:6789/0 conn(0x7efec413ede0 legacy=0x7efec40b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 1 -- 172.21.15.174:0/976100372 >> v1:172.21.15.174:6789/0 conn(0x7efec40b0bd0 legacy=0x7efec414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:32.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 10 monclient: _finish_hunting 0 2023-11-02T16:13:32.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 1 monclient: found mon.noname-c 2023-11-02T16:13:32.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:32.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.226+0000 7efec3fff640 1 -- 172.21.15.174:0/976100372 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40afbc0 con 0x7efec40b3c10 2023-11-02T16:13:32.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.229+0000 7efec3fff640 1 --2- 172.21.15.174:0/976100372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40b3c10 0x7efec413e8a0 secure :-1 s=READY pgs=5277 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097a0 tx=0x7efeb802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d16d8fa79abef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 1 -- 172.21.15.174:0/976100372 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb802e070 con 0x7efec40b3c10 2023-11-02T16:13:32.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:32.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: dump: 2023-11-02T16:13:32.237 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.237 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.237 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.238 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.238 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.238 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.238 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.239 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.239 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.239 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:32.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 1 -- 172.21.15.174:0/976100372 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb80315a0 con 0x7efec40b3c10 2023-11-02T16:13:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efeca71b640 5 monclient: authenticate success, global_id 21574 2023-11-02T16:13:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efeca71b640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:32.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efeca71b640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:32.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 1 -- 172.21.15.174:0/976100372 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8031930 con 0x7efec40b3c10 2023-11-02T16:13:32.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efeca71b640 4 set_mon_vals no callback set 2023-11-02T16:13:32.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.241 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:32.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.234+0000 7efec2ffd640 10 monclient: dump: 2023-11-02T16:13:32.242 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.242 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.242 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.243 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.243 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.243 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.243 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.243 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.244 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.244 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 10 monclient: shutdown 2023-11-02T16:13:32.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:32.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 -- 172.21.15.174:0/976100372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40b3c10 msgr2=0x7efec413e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 --2- 172.21.15.174:0/976100372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40b3c10 0x7efec413e8a0 secure :-1 s=READY pgs=5277 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097a0 tx=0x7efeb802b840 comp rx=0 tx=0).stop 2023-11-02T16:13:32.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 -- 172.21.15.174:0/976100372 shutdown_connections 2023-11-02T16:13:32.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 --2- 172.21.15.174:0/976100372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec40b3c10 0x7efec413e8a0 unknown :-1 s=CLOSED pgs=5277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.248 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 -- 172.21.15.174:0/976100372 >> 172.21.15.174:0/976100372 conn(0x7efec4065960 msgr2=0x7efec4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:32.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 -- 172.21.15.174:0/976100372 shutdown_connections 2023-11-02T16:13:32.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.235+0000 7efeca71b640 1 -- 172.21.15.174:0/976100372 wait complete. 2023-11-02T16:13:32.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.236+0000 7efeca71b640 10 monclient: build_initial_monmap 2023-11-02T16:13:32.249 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.236+0000 7efeca71b640 10 monclient: monmap: 2023-11-02T16:13:32.249 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:32.250 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:32.250 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:32.250 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:32.250 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:32.251 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.251 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:32.251 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:32.251 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:32.252 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.236+0000 7efeca71b640 1 librados: starting msgr at 2023-11-02T16:13:32.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.236+0000 7efeca71b640 1 librados: starting objecter 2023-11-02T16:13:32.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.236+0000 7efeca71b640 1 Processor -- start 2023-11-02T16:13:32.252 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 -- start start 2023-11-02T16:13:32.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 librados: setting wanted keys 2023-11-02T16:13:32.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 librados: calling monclient init 2023-11-02T16:13:32.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient: init 2023-11-02T16:13:32.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:32.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T16:13:32.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efec40b0bd0 0x7efec41436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-c con 0x7efec40b0bd0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:32.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec40b3c10 0x7efec4141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.254 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-a con 0x7efec40b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:32.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 0x7efec41361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): picked mon.noname-b con 0x7efec413ede0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:32.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.255 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4144490 con 0x7efec40b3c10 2023-11-02T16:13:32.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4141a20 con 0x7efec413ede0 2023-11-02T16:13:32.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.256 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec4140960 con 0x7efec40b0bd0 2023-11-02T16:13:32.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:32.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.237+0000 7efeca71b640 10 monclient(hunting): authenticate will time out at 3739.579256s 2023-11-02T16:13:32.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 0x7efec41361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 0x7efec41361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54940/0 (socket says 172.21.15.174:54940) 2023-11-02T16:13:32.257 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 1 -- 172.21.15.174:0/3320267118 learned_addr learned my addr 172.21.15.174:0/3320267118 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:32.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 10 monclient(hunting): get_auth_request con 0x7efec413ede0 auth_method 0 2023-11-02T16:13:32.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:32.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:32.258 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.238+0000 7efec8cb8640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:32.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.239+0000 7efec8cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:32.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.239+0000 7efec8cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:32.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.239+0000 7efec8cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:32.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 10 monclient(hunting): handle_auth_done global_id 21580 payload 824 2023-11-02T16:13:32.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 -- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efec40b0bd0 msgr2=0x7efec41436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efec40b0bd0 0x7efec41436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 -- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec40b3c10 msgr2=0x7efec4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec40b3c10 0x7efec4141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 10 monclient: _finish_hunting 0 2023-11-02T16:13:32.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 monclient: found mon.noname-b 2023-11-02T16:13:32.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:32.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.241+0000 7efec8cb8640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8009400 con 0x7efec413ede0 2023-11-02T16:13:32.261 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.243+0000 7efec8cb8640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 0x7efec41361d0 secure :-1 s=READY pgs=5278 cs=0 l=1 rev1=1 crypto rx=0x7efebc00cb20 tx=0x7efebc00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.243+0000 7efec0ff9640 10 client.?.objecter ms_handle_connect 0x7efec413ede0 2023-11-02T16:13:32.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.243+0000 7efec0ff9640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:32.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00aca0 con 0x7efec413ede0 2023-11-02T16:13:32.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:32.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: dump: 2023-11-02T16:13:32.264 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.264 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.265 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.265 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.265 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.265 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.266 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.266 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.266 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.266 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: _finish_auth 0 2023-11-02T16:13:32.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: _check_auth_tickets 2023-11-02T16:13:32.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:32.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00ae20 con 0x7efec413ede0 2023-11-02T16:13:32.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:32.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc012640 con 0x7efec413ede0 2023-11-02T16:13:32.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 5 monclient: authenticate success, global_id 21580 2023-11-02T16:13:32.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 10 monclient: _renew_subs 2023-11-02T16:13:32.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4140b80 con 0x7efec413ede0 2023-11-02T16:13:32.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec27fc640 4 set_mon_vals no callback set 2023-11-02T16:13:32.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:32.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efec0ff9640 10 monclient: dump: 2023-11-02T16:13:32.269 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.270 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.270 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.270 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.270 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.270 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.271 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.271 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.271 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.271 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 10 client.21580.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:32.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 10 monclient: _renew_subs 2023-11-02T16:13:32.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec41368f0 con 0x7efec413ede0 2023-11-02T16:13:32.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.247+0000 7efeca71b640 1 librados: init done 2023-11-02T16:13:32.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 10 client.21580.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:32.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 15 client.21580.objecter _session_command_op_assign -1 1 2023-11-02T16:13:32.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 20 client.21580.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:32.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 10 client.21580.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:32.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.248+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efec41418b0 con 0x7efec413ede0 2023-11-02T16:13:32.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.266+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efebc0127a0 con 0x7efec413ede0 2023-11-02T16:13:32.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.266+0000 7efec0ff9640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7efeb006bdc0 0x7efeb006e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.267+0000 7efec3fff640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7efeb006bdc0 0x7efeb006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.267+0000 7efec3fff640 10 monclient: get_auth_request con 0x7efeb006bdc0 auth_method 0 2023-11-02T16:13:32.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.267+0000 7efec0ff9640 10 client.21580.objecter ms_handle_connect 0x7efeb006bdc0 2023-11-02T16:13:32.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.267+0000 7efec3fff640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7efeb006bdc0 0x7efeb006e280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efeb8036000 tx=0x7efeb8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7efebc059350 con 0x7efec413ede0 2023-11-02T16:13:32.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 10 client.21580.objecter ms_dispatch 0x7efec4000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:32.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 3 client.21580.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:32.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 3 client.21580.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:32.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 10 client.21580.objecter checking command 1 2023-11-02T16:13:32.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7efeb0073730 0x7efeb0075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter _get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.278 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 15 client.21580.objecter _session_command_op_remove -1 1 2023-11-02T16:13:32.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter _get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 15 client.21580.objecter _session_command_op_assign 6 1 2023-11-02T16:13:32.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 4 2023-11-02T16:13:32.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 10 client.21580.objecter _send_command 1 2023-11-02T16:13:32.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efeb00762a0 con 0x7efeb0073730 2023-11-02T16:13:32.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.268+0000 7efec0ff9640 20 client.21580.objecter dump_active .. 0 homeless 2023-11-02T16:13:32.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.269+0000 7efec37fe640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7efeb0073730 0x7efeb0075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.269+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7efebc054f90 con 0x7efec413ede0 2023-11-02T16:13:32.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.269+0000 7efec0ff9640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:32.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.269+0000 7efec37fe640 10 monclient: get_auth_request con 0x7efeb0073730 auth_method 0 2023-11-02T16:13:32.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.272+0000 7efec37fe640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7efeb0073730 0x7efeb0075bd0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.272+0000 7efec0ff9640 10 client.21580.objecter ms_handle_connect 0x7efeb0073730 2023-11-02T16:13:32.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.295+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== osd.6 v2:172.21.15.175:6824/86417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efeb00762a0 con 0x7efeb0073730 2023-11-02T16:13:32.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.295+0000 7efec0ff9640 10 client.21580.objecter ms_dispatch 0x7efec4000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:32.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.295+0000 7efec0ff9640 10 client.21580.objecter _finish_command 1 = system:0 2023-11-02T16:13:32.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.295+0000 7efec0ff9640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 4 2023-11-02T16:13:32.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.295+0000 7efec0ff9640 15 client.21580.objecter _session_command_op_remove 6 1 2023-11-02T16:13:32.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 10 client.21580.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:32.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 15 client.21580.objecter _session_command_op_assign -1 2 2023-11-02T16:13:32.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 20 client.21580.objecter _get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 20 client.21580.objecter _get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 15 client.21580.objecter _session_command_op_remove -1 2 2023-11-02T16:13:32.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 20 client.21580.objecter get_session s=0x7efeb0073060 osd=6 3 2023-11-02T16:13:32.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 15 client.21580.objecter _session_command_op_assign 6 2 2023-11-02T16:13:32.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 4 2023-11-02T16:13:32.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 10 client.21580.objecter _send_command 2 2023-11-02T16:13:32.313 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.308+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 --> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efec40ae380 con 0x7efeb0073730 2023-11-02T16:13:32.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec0ff9640 1 -- 172.21.15.174:0/3320267118 <== osd.6 v2:172.21.15.175:6824/86417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efeb4001190 con 0x7efeb0073730 2023-11-02T16:13:32.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec0ff9640 10 client.21580.objecter ms_dispatch 0x7efec4000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:32.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec0ff9640 10 client.21580.objecter _finish_command 2 = system:0 2023-11-02T16:13:32.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec0ff9640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 4 2023-11-02T16:13:32.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec0ff9640 15 client.21580.objecter _session_command_op_remove 6 2 2023-11-02T16:13:32.325 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:32.326 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:32.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 10 librados: watch_flush enter 2023-11-02T16:13:32.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 10 librados: watch_flush exit 2023-11-02T16:13:32.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 10 client.21580.objecter close_session for osd.6 2023-11-02T16:13:32.328 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7efeb0073730 msgr2=0x7efeb0075bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:6824/86417,v1:172.21.15.175:6825/86417] conn(0x7efeb0073730 0x7efeb0075bd0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 20 client.21580.objecter put_session s=0x7efeb0073060 osd=6 1 2023-11-02T16:13:32.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 20 client.21580.objecter shutdown clearing up homeless session... 2023-11-02T16:13:32.329 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 10 client.21580.objecter successfully canceled tick 2023-11-02T16:13:32.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7efeb006bdc0 msgr2=0x7efeb006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7efeb006bdc0 0x7efeb006e280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efeb8036000 tx=0x7efeb8030000 comp rx=0 tx=0).stop 2023-11-02T16:13:32.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 10 monclient: shutdown 2023-11-02T16:13:32.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:32.330 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 msgr2=0x7efec41361d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 --2- 172.21.15.174:0/3320267118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec413ede0 0x7efec41361d0 secure :-1 s=READY pgs=5278 cs=0 l=1 rev1=1 crypto rx=0x7efebc00cb20 tx=0x7efebc00a060 comp rx=0 tx=0).stop 2023-11-02T16:13:32.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efec8cb8640 1 -- 172.21.15.174:0/3320267118 reap_dead start 2023-11-02T16:13:32.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 shutdown_connections 2023-11-02T16:13:32.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.322+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 >> 172.21.15.174:0/3320267118 conn(0x7efec4065960 msgr2=0x7efec4073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:32.331 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.323+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 shutdown_connections 2023-11-02T16:13:32.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.323+0000 7efeca71b640 1 -- 172.21.15.174:0/3320267118 wait complete. 2023-11-02T16:13:32.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.323+0000 7efeca71b640 1 librados: shutdown 2023-11-02T16:13:32.336 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-02T16:13:32.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.452+0000 7fc778739640 10 monclient: get_monmap_and_config 2023-11-02T16:13:32.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.452+0000 7fc778739640 10 monclient: build_initial_monmap 2023-11-02T16:13:32.454 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.452+0000 7fc778739640 10 monclient: monmap: 2023-11-02T16:13:32.455 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:32.455 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.455 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:32.453370+0000 2023-11-02T16:13:32.455 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:32.453370+0000 2023-11-02T16:13:32.455 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:32.456 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.456 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:32.456 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:32.456 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:32.456 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:32.457 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:32.457 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 Processor -- start 2023-11-02T16:13:32.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 -- start start 2023-11-02T16:13:32.459 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient: init 2023-11-02T16:13:32.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:32.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient: _add_conns ranks=[1,2,0,4,3] 2023-11-02T16:13:32.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc770065bd0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:32.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 0x7fc77013ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): picked mon.noname-b con 0x7fc77013ea50 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7700a8660 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7700aa1d0 con 0x7fc7700a8660 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7700aa440 con 0x7fc770065bd0 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc770145c00 con 0x7fc77013ea50 2023-11-02T16:13:32.461 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:32.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.453+0000 7fc778739640 10 monclient(hunting): authenticate will time out at 3739.795510s 2023-11-02T16:13:32.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc775cd4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 0x7fc77013ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc7764d5640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fc770065bd0 0x7fc770145270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51454/0 (socket says 172.21.15.174:51454) 2023-11-02T16:13:32.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc7764d5640 1 -- 172.21.15.174:0/902749272 learned_addr learned my addr 172.21.15.174:0/902749272 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:32.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc775cd4640 10 monclient(hunting): get_auth_request con 0x7fc77013ea50 auth_method 0 2023-11-02T16:13:32.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc775cd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:32.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc775cd4640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:32.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.454+0000 7fc775cd4640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:32.464 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.455+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:32.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.455+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:32.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.455+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:32.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 10 monclient(hunting): handle_auth_done global_id 16955 payload 274 2023-11-02T16:13:32.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc7754d3640 1 -- 172.21.15.174:0/902749272 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204457805 0 0) 0x7fc7700aa440 con 0x7fc770065bd0 2023-11-02T16:13:32.465 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 1 -- 172.21.15.174:0/902749272 >> v1:172.21.15.175:6789/0 conn(0x7fc770065bd0 legacy=0x7fc770145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 1 -- 172.21.15.174:0/902749272 >> v1:172.21.15.174:6789/0 conn(0x7fc7700a8660 legacy=0x7fc7700a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:32.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 10 monclient: _finish_hunting 0 2023-11-02T16:13:32.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 1 monclient: found mon.noname-b 2023-11-02T16:13:32.466 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:32.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:32.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.456+0000 7fc775cd4640 1 -- 172.21.15.174:0/902749272 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77013f3b0 con 0x7fc77013ea50 2023-11-02T16:13:32.467 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.457+0000 7fc7754d3640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-11-02T16:13:32.470 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.459+0000 7fc775cd4640 1 --2- 172.21.15.174:0/902749272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 0x7fc77013ee70 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097d0 tx=0x7fc76002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22479abd8ce259bd server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 1 -- 172.21.15.174:0/902749272 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600316f0 con 0x7fc77013ea50 2023-11-02T16:13:32.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:32.471 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: dump: 2023-11-02T16:13:32.472 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.472 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.472 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.472 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.473 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: _finish_auth 0 2023-11-02T16:13:32.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: _check_auth_tickets 2023-11-02T16:13:32.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:32.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 1 -- 172.21.15.174:0/902749272 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760031850 con 0x7fc77013ea50 2023-11-02T16:13:32.474 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:32.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 1 -- 172.21.15.174:0/902749272 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760031be0 con 0x7fc77013ea50 2023-11-02T16:13:32.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc778739640 5 monclient: authenticate success, global_id 16955 2023-11-02T16:13:32.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc778739640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:32.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc778739640 4 set_mon_vals no callback set 2023-11-02T16:13:32.475 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:32.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.463+0000 7fc7754d3640 10 monclient: dump: 2023-11-02T16:13:32.476 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.476 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.476 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.477 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.478 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 10 monclient: shutdown 2023-11-02T16:13:32.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:32.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 -- 172.21.15.174:0/902749272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 msgr2=0x7fc77013ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 --2- 172.21.15.174:0/902749272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 0x7fc77013ee70 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097d0 tx=0x7fc76002b790 comp rx=0 tx=0).stop 2023-11-02T16:13:32.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 -- 172.21.15.174:0/902749272 shutdown_connections 2023-11-02T16:13:32.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 --2- 172.21.15.174:0/902749272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc77013ea50 0x7fc77013ee70 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 -- 172.21.15.174:0/902749272 >> 172.21.15.174:0/902749272 conn(0x7fc77006a830 msgr2=0x7fc7700b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:32.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 -- 172.21.15.174:0/902749272 shutdown_connections 2023-11-02T16:13:32.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.464+0000 7fc778739640 1 -- 172.21.15.174:0/902749272 wait complete. 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 10 monclient: build_initial_monmap 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 10 monclient: monmap: 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:32.480 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:32.482 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 librados: starting msgr at 2023-11-02T16:13:32.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 librados: starting objecter 2023-11-02T16:13:32.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 Processor -- start 2023-11-02T16:13:32.482 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 -- start start 2023-11-02T16:13:32.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 librados: setting wanted keys 2023-11-02T16:13:32.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 1 librados: calling monclient init 2023-11-02T16:13:32.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.465+0000 7fc778739640 10 monclient: init 2023-11-02T16:13:32.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:32.483 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): picked mon.noname-b con 0x7fc770065bd0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7700a8660 0x7fc77013a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): picked mon.noname-c con 0x7fc7700a8660 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77013ea50 0x7fc77012d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.484 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): picked mon.noname-a con 0x7fc77013ea50 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:32.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77013b360 con 0x7fc77013ea50 2023-11-02T16:13:32.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770134ba0 con 0x7fc770065bd0 2023-11-02T16:13:32.485 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:32.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77013b5c0 con 0x7fc7700a8660 2023-11-02T16:13:32.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:32.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc778739640 10 monclient(hunting): authenticate will time out at 3739.808170s 2023-11-02T16:13:32.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.466+0000 7fc775cd4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7700a8660 0x7fc77013a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.486 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54950/0 (socket says 172.21.15.174:54950) 2023-11-02T16:13:32.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 1 -- 172.21.15.174:0/2714581950 learned_addr learned my addr 172.21.15.174:0/2714581950 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:32.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): get_auth_request con 0x7fc770065bd0 auth_method 0 2023-11-02T16:13:32.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:32.487 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:32.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:32.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc775cd4640 10 monclient(hunting): get_auth_request con 0x7fc7700a8660 auth_method 0 2023-11-02T16:13:32.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc775cd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:32.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc775cd4640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:32.488 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc775cd4640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:32.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:32.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:32.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.467+0000 7fc7764d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:32.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.468+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:32.489 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.468+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:32.490 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.468+0000 7fc775cd4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:32.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 10 monclient(hunting): handle_auth_done global_id 21586 payload 824 2023-11-02T16:13:32.492 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 -- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7700a8660 msgr2=0x7fc77013a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7700a8660 0x7fc77013a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 -- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77013ea50 msgr2=0x7fc77012d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77013ea50 0x7fc77012d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 10 monclient: _finish_hunting 0 2023-11-02T16:13:32.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 monclient: found mon.noname-b 2023-11-02T16:13:32.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:32.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.470+0000 7fc7764d5640 1 -- 172.21.15.174:0/2714581950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009400 con 0x7fc770065bd0 2023-11-02T16:13:32.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.474+0000 7fc7764d5640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 secure :-1 s=READY pgs=5280 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c4d0 tx=0x7fc76c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.474+0000 7fc766ffd640 10 client.?.objecter ms_handle_connect 0x7fc770065bd0 2023-11-02T16:13:32.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.474+0000 7fc766ffd640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:32.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c0074e0 con 0x7fc770065bd0 2023-11-02T16:13:32.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:32.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: dump: 2023-11-02T16:13:32.496 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.496 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.496 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.497 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.497 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.497 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.497 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.497 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.498 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.498 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: _finish_auth 0 2023-11-02T16:13:32.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: _check_auth_tickets 2023-11-02T16:13:32.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:32.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c007660 con 0x7fc770065bd0 2023-11-02T16:13:32.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 5 monclient: authenticate success, global_id 21586 2023-11-02T16:13:32.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 10 monclient: _renew_subs 2023-11-02T16:13:32.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7701418f0 con 0x7fc770065bd0 2023-11-02T16:13:32.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:32.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004a20 con 0x7fc770065bd0 2023-11-02T16:13:32.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:32.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:32.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc766ffd640 10 monclient: dump: 2023-11-02T16:13:32.501 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:32.501 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:32.501 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.502 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:32.502 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:32.502 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:32.502 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:32.502 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:32.503 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:32.503 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:32.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc774cd2640 4 set_mon_vals no callback set 2023-11-02T16:13:32.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 10 client.21586.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:32.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 10 monclient: _renew_subs 2023-11-02T16:13:32.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77013bf20 con 0x7fc770065bd0 2023-11-02T16:13:32.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.479+0000 7fc778739640 1 librados: init done 2023-11-02T16:13:32.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.480+0000 7fc778739640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:32.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.480+0000 7fc778739640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:32.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.480+0000 7fc778739640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:32.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.480+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc770141b30 con 0x7fc770065bd0 2023-11-02T16:13:32.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.499+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc76c01b360 con 0x7fc770065bd0 2023-11-02T16:13:32.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.499+0000 7fc766ffd640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 0x7fc75406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:32.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.499+0000 7fc775cd4640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 0x7fc75406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:32.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.499+0000 7fc775cd4640 10 monclient: get_auth_request con 0x7fc75406bdc0 auth_method 0 2023-11-02T16:13:32.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.500+0000 7fc766ffd640 10 client.21586.objecter ms_handle_connect 0x7fc75406bdc0 2023-11-02T16:13:32.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.500+0000 7fc775cd4640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 0x7fc75406e280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc760036000 tx=0x7fc760008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:32.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.501+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc76c056f60 con 0x7fc770065bd0 2023-11-02T16:13:32.508 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.501+0000 7fc766ffd640 10 client.21586.objecter ms_dispatch 0x7fc770000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:32.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.501+0000 7fc766ffd640 3 client.21586.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:32.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.501+0000 7fc766ffd640 3 client.21586.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:32.509 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.501+0000 7fc766ffd640 20 client.21586.objecter dump_active .. 0 homeless 2023-11-02T16:13:32.758 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.757+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc76c0532a0 con 0x7fc770065bd0 2023-11-02T16:13:32.759 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.757+0000 7fc766ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:32.759 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.757+0000 7fc766ffd640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:32.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.954+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc770065fb0 con 0x7fc75406bdc0 2023-11-02T16:13:32.956 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.954+0000 7fc766ffd640 1 -- 172.21.15.174:0/2714581950 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc760031b40 con 0x7fc75406bdc0 2023-11-02T16:13:32.956 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 6 to deep-scrub 2023-11-02T16:13:32.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 10 librados: watch_flush enter 2023-11-02T16:13:32.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 10 librados: watch_flush exit 2023-11-02T16:13:32.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 20 client.21586.objecter shutdown clearing up homeless session... 2023-11-02T16:13:32.959 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 10 client.21586.objecter successfully canceled tick 2023-11-02T16:13:32.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 msgr2=0x7fc75406e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 0x7fc75406e280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc760036000 tx=0x7fc760008030 comp rx=0 tx=0).stop 2023-11-02T16:13:32.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 10 monclient: shutdown 2023-11-02T16:13:32.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:32.960 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 msgr2=0x7fc770139e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:32.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 secure :-1 s=READY pgs=5280 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c4d0 tx=0x7fc76c00c990 comp rx=0 tx=0).stop 2023-11-02T16:13:32.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 shutdown_connections 2023-11-02T16:13:32.961 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77013ea50 0x7fc77012d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fc75406bdc0 0x7fc75406e280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7700a8660 0x7fc77013a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 --2- 172.21.15.174:0/2714581950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc770065bd0 0x7fc770139e10 unknown :-1 s=CLOSED pgs=5280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 >> 172.21.15.174:0/2714581950 conn(0x7fc77006a830 msgr2=0x7fc7700b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 shutdown_connections 2023-11-02T16:13:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 -- 172.21.15.174:0/2714581950 wait complete. 2023-11-02T16:13:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:32.956+0000 7fc778739640 1 librados: shutdown 2023-11-02T16:13:32.970 INFO:tasks.ceph:Scrubbing osd.7 2023-11-02T16:13:32.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-02T16:13:33.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.077+0000 7f2ea6192640 10 monclient: get_monmap_and_config 2023-11-02T16:13:33.079 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.077+0000 7f2ea6192640 10 monclient: build_initial_monmap 2023-11-02T16:13:33.080 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.077+0000 7f2ea6192640 10 monclient: monmap: 2023-11-02T16:13:33.080 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:33.080 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.080 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:33.078687+0000 2023-11-02T16:13:33.081 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:33.078687+0000 2023-11-02T16:13:33.081 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:33.081 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.081 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:33.081 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:33.082 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:33.082 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:33.082 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:33.082 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.078+0000 7f2ea6192640 1 Processor -- start 2023-11-02T16:13:33.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.078+0000 7f2ea6192640 1 -- start start 2023-11-02T16:13:33.084 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.078+0000 7f2ea6192640 10 monclient: init 2023-11-02T16:13:33.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:33.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient: _add_conns ranks=[2,3,0,1,4] 2023-11-02T16:13:33.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-b con 0x7f2ea00b3c10 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:33.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2ea013ede0 0x7f2ea00b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-a con 0x7f2ea013ede0 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:33.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2ea00b0a00 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:33.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea00af970 con 0x7f2ea00b0a00 2023-11-02T16:13:33.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea00b3ff0 con 0x7f2ea013ede0 2023-11-02T16:13:33.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea00afbc0 con 0x7f2ea00b3c10 2023-11-02T16:13:33.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:33.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.079+0000 7f2ea6192640 10 monclient(hunting): authenticate will time out at 3740.420931s 2023-11-02T16:13:33.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.081+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54788/0 (socket says 172.21.15.174:54788) 2023-11-02T16:13:33.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/1990883127 learned_addr learned my addr 172.21.15.174:0/1990883127 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:33.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 10 monclient(hunting): get_auth_request con 0x7f2ea00b3c10 auth_method 0 2023-11-02T16:13:33.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:33.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.082+0000 7f2e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.083+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.083+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.083+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_done global_id 16961 payload 274 2023-11-02T16:13:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/1990883127 >> v2:172.21.15.174:3300/0 conn(0x7f2ea013ede0 msgr2=0x7f2ea00b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 1 --2- 172.21.15.174:0/1990883127 >> v2:172.21.15.174:3300/0 conn(0x7f2ea013ede0 0x7f2ea00b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/1990883127 >> v1:172.21.15.174:6789/0 conn(0x7f2ea00b0a00 legacy=0x7f2ea00b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:33.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 1 monclient: found mon.noname-b 2023-11-02T16:13:33.092 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:33.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:33.093 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.085+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/1990883127 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea00afea0 con 0x7f2ea00b3c10 2023-11-02T16:13:33.095 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.087+0000 7f2e9f7fe640 1 --2- 172.21.15.174:0/1990883127 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009e50 tx=0x7f2e9002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecada2b8c47937bd server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 1 -- 172.21.15.174:0/1990883127 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9002c9e0 con 0x7f2ea00b3c10 2023-11-02T16:13:33.099 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:33.100 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 10 monclient: dump: 2023-11-02T16:13:33.100 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.100 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.101 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.101 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.101 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.101 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.101 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.102 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.102 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.102 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:33.102 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:33.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2e9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:33.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2ea6192640 5 monclient: authenticate success, global_id 16961 2023-11-02T16:13:33.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.090+0000 7f2ea6192640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:33.103 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 1 -- 172.21.15.174:0/1990883127 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9002cb40 con 0x7f2ea00b3c10 2023-11-02T16:13:33.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:33.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 1 -- 172.21.15.174:0/1990883127 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900343f0 con 0x7f2ea00b3c10 2023-11-02T16:13:33.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2ea6192640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:33.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2ea6192640 4 set_mon_vals no callback set 2023-11-02T16:13:33.104 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:33.105 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.092+0000 7f2e9e7fc640 10 monclient: dump: 2023-11-02T16:13:33.105 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.106 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.106 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.106 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.107 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.107 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.108 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.108 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.108 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.109 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 10 monclient: shutdown 2023-11-02T16:13:33.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:33.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 -- 172.21.15.174:0/1990883127 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 msgr2=0x7f2ea013e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 --2- 172.21.15.174:0/1990883127 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009e50 tx=0x7f2e9002be40 comp rx=0 tx=0).stop 2023-11-02T16:13:33.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 -- 172.21.15.174:0/1990883127 shutdown_connections 2023-11-02T16:13:33.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 --2- 172.21.15.174:0/1990883127 >> v2:172.21.15.174:3300/0 conn(0x7f2ea013ede0 0x7f2ea00b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 --2- 172.21.15.174:0/1990883127 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b3c10 0x7f2ea013e8a0 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 -- 172.21.15.174:0/1990883127 >> 172.21.15.174:0/1990883127 conn(0x7f2ea0065960 msgr2=0x7f2ea0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:33.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 -- 172.21.15.174:0/1990883127 shutdown_connections 2023-11-02T16:13:33.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 1 -- 172.21.15.174:0/1990883127 wait complete. 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 10 monclient: build_initial_monmap 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.093+0000 7f2ea6192640 10 monclient: monmap: 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:33.113 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:33.114 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:33.114 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.114 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:33.114 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:33.115 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:33.115 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.115 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 librados: starting msgr at 2023-11-02T16:13:33.116 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 librados: starting objecter 2023-11-02T16:13:33.116 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 Processor -- start 2023-11-02T16:13:33.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 -- start start 2023-11-02T16:13:33.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 librados: setting wanted keys 2023-11-02T16:13:33.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 1 librados: calling monclient init 2023-11-02T16:13:33.117 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.094+0000 7f2ea6192640 10 monclient: init 2023-11-02T16:13:33.118 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:33.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient: _add_conns ranks=[2,0,1] 2023-11-02T16:13:33.119 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 0x7f2ea013c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-c con 0x7f2ea00b0a00 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:33.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea00b3c10 0x7f2ea0143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-a con 0x7f2ea00b3c10 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:33.120 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea013ede0 0x7f2ea01da940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): picked mon.noname-b con 0x7f2ea013ede0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:33.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01da180 con 0x7f2ea00b3c10 2023-11-02T16:13:33.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01da420 con 0x7f2ea013ede0 2023-11-02T16:13:33.122 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01431f0 con 0x7f2ea00b0a00 2023-11-02T16:13:33.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:33.123 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2ea6192640 10 monclient(hunting): authenticate will time out at 3740.436881s 2023-11-02T16:13:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 0x7f2ea013c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.095+0000 7f2e9ffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea013ede0 0x7f2ea01da940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 0x7f2ea013c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54800/0 (socket says 172.21.15.174:54800) 2023-11-02T16:13:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/4148813429 learned_addr learned my addr 172.21.15.174:0/4148813429 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): get_auth_request con 0x7f2ea00b0a00 auth_method 0 2023-11-02T16:13:33.125 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:33.125 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.125 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:33.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): get_auth_request con 0x7f2ea013ede0 auth_method 0 2023-11-02T16:13:33.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:33.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:33.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:33.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:33.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:33.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:33.127 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:33.128 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.096+0000 7f2e9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:33.130 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 10 monclient(hunting): handle_auth_done global_id 16964 payload 824 2023-11-02T16:13:33.130 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea013ede0 msgr2=0x7f2ea01da940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.130 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ea013ede0 0x7f2ea01da940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea00b3c10 msgr2=0x7f2ea0143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea00b3c10 0x7f2ea0143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:33.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 monclient: found mon.noname-c 2023-11-02T16:13:33.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:33.132 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:13:33.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.099+0000 7f2e9f7fe640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e90009b30 con 0x7f2ea00b0a00 2023-11-02T16:13:33.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.101+0000 7f2e9f7fe640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 0x7f2ea013c400 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009e50 tx=0x7f2e90004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.101+0000 7f2e87fff640 10 client.?.objecter ms_handle_connect 0x7f2ea00b0a00 2023-11-02T16:13:33.133 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.101+0000 7f2e87fff640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90009510 con 0x7f2ea00b0a00 2023-11-02T16:13:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:13:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: dump: 2023-11-02T16:13:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.135 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.135 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.135 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.135 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.135 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.136 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.136 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.136 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.136 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: _finish_auth 0 2023-11-02T16:13:33.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: _check_auth_tickets 2023-11-02T16:13:33.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:33.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9003e450 con 0x7f2ea00b0a00 2023-11-02T16:13:33.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 5 monclient: authenticate success, global_id 16964 2023-11-02T16:13:33.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 10 monclient: _renew_subs 2023-11-02T16:13:33.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:33.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea0144640 con 0x7f2ea00b0a00 2023-11-02T16:13:33.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:33.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9003e800 con 0x7f2ea00b0a00 2023-11-02T16:13:33.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:33.140 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e87fff640 10 monclient: dump: 2023-11-02T16:13:33.140 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.140 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.140 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.140 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.141 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.141 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.141 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.141 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.141 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.142 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 10 client.16964.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:33.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 10 monclient: _renew_subs 2023-11-02T16:13:33.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2e9dffb640 4 set_mon_vals no callback set 2023-11-02T16:13:33.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:33.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea01db060 con 0x7f2ea00b0a00 2023-11-02T16:13:33.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.104+0000 7f2ea6192640 1 librados: init done 2023-11-02T16:13:33.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 10 client.16964.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:33.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 15 client.16964.objecter _session_command_op_assign -1 1 2023-11-02T16:13:33.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 20 client.16964.objecter _get_session osd=-1 returning homeless 2023-11-02T16:13:33.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 10 client.16964.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:33.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:13:33.145 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.105+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ea013f220 con 0x7f2ea00b0a00 2023-11-02T16:13:33.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.147+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2e9003e960 con 0x7f2ea00b0a00 2023-11-02T16:13:33.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.148+0000 7f2e87fff640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2e8c06bdc0 0x7f2e8c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.149+0000 7f2e9effd640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2e8c06bdc0 0x7f2e8c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.151 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.149+0000 7f2e9effd640 10 monclient: get_auth_request con 0x7f2e8c06bdc0 auth_method 0 2023-11-02T16:13:33.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.149+0000 7f2e87fff640 10 client.16964.objecter ms_handle_connect 0x7f2e8c06bdc0 2023-11-02T16:13:33.152 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.149+0000 7f2e9effd640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2e8c06bdc0 0x7f2e8c06e280 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ea013dd00 tx=0x7f2e94006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f2e9007b660 con 0x7f2ea00b0a00 2023-11-02T16:13:33.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 10 client.16964.objecter ms_dispatch 0x7f2ea0000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:33.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 3 client.16964.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:33.156 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 3 client.16964.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:33.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 10 client.16964.objecter checking command 1 2023-11-02T16:13:33.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f2e8c073730 0x7f2e8c075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter _get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.157 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 15 client.16964.objecter _session_command_op_remove -1 1 2023-11-02T16:13:33.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter _get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 15 client.16964.objecter _session_command_op_assign 7 1 2023-11-02T16:13:33.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 4 2023-11-02T16:13:33.158 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 10 client.16964.objecter _send_command 1 2023-11-02T16:13:33.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e8c0762a0 con 0x7f2e8c073730 2023-11-02T16:13:33.159 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.154+0000 7f2e87fff640 20 client.16964.objecter dump_active .. 0 homeless 2023-11-02T16:13:33.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.155+0000 7f2e9ffff640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f2e8c073730 0x7f2e8c075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.155+0000 7f2e9ffff640 10 monclient: get_auth_request con 0x7f2e8c073730 auth_method 0 2023-11-02T16:13:33.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.156+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=1030) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e900772a0 con 0x7f2ea00b0a00 2023-11-02T16:13:33.160 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.156+0000 7f2e87fff640 10 monclient: handle_get_version_reply finishing 1 version 1030 2023-11-02T16:13:33.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.158+0000 7f2e87fff640 10 client.16964.objecter ms_handle_connect 0x7f2e8c073730 2023-11-02T16:13:33.161 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.158+0000 7f2e9ffff640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f2e8c073730 0x7f2e8c075bd0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.196 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.194+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== osd.7 v2:172.21.15.175:6800/86418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2e8c0762a0 con 0x7f2e8c073730 2023-11-02T16:13:33.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.194+0000 7f2e87fff640 10 client.16964.objecter ms_dispatch 0x7f2ea0000b90 command_reply(tid 1: 0 ) v1 2023-11-02T16:13:33.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.194+0000 7f2e87fff640 10 client.16964.objecter _finish_command 1 = system:0 2023-11-02T16:13:33.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.194+0000 7f2e87fff640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 4 2023-11-02T16:13:33.197 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.194+0000 7f2e87fff640 15 client.16964.objecter _session_command_op_remove 7 1 2023-11-02T16:13:33.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 10 client.16964.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2023-11-02T16:13:33.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 15 client.16964.objecter _session_command_op_assign -1 2 2023-11-02T16:13:33.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 20 client.16964.objecter _get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.216 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 20 client.16964.objecter _get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 15 client.16964.objecter _session_command_op_remove -1 2 2023-11-02T16:13:33.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 20 client.16964.objecter get_session s=0x7f2e8c073060 osd=7 3 2023-11-02T16:13:33.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 15 client.16964.objecter _session_command_op_assign 7 2 2023-11-02T16:13:33.217 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 4 2023-11-02T16:13:33.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 10 client.16964.objecter _send_command 2 2023-11-02T16:13:33.218 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.214+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 --> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ea00ae380 con 0x7f2e8c073730 2023-11-02T16:13:33.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e87fff640 1 -- 172.21.15.174:0/4148813429 <== osd.7 v2:172.21.15.175:6800/86418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2e980013c0 con 0x7f2e8c073730 2023-11-02T16:13:33.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e87fff640 10 client.16964.objecter ms_dispatch 0x7f2ea0000b90 command_reply(tid 2: 0 ) v1 2023-11-02T16:13:33.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e87fff640 10 client.16964.objecter _finish_command 2 = system:0 2023-11-02T16:13:33.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e87fff640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 4 2023-11-02T16:13:33.239 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e87fff640 15 client.16964.objecter _session_command_op_remove 7 2 2023-11-02T16:13:33.239 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-02T16:13:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-02T16:13:33.239 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-02T16:13:33.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 10 librados: watch_flush enter 2023-11-02T16:13:33.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 10 librados: watch_flush exit 2023-11-02T16:13:33.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 10 client.16964.objecter close_session for osd.7 2023-11-02T16:13:33.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f2e8c073730 msgr2=0x7f2e8c075bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.242 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.175:6800/86418,v1:172.21.15.175:6801/86418] conn(0x7f2e8c073730 0x7f2e8c075bd0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 20 client.16964.objecter put_session s=0x7f2e8c073060 osd=7 1 2023-11-02T16:13:33.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 20 client.16964.objecter shutdown clearing up homeless session... 2023-11-02T16:13:33.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 10 client.16964.objecter successfully canceled tick 2023-11-02T16:13:33.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2e8c06bdc0 msgr2=0x7f2e8c06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.243 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2e8c06bdc0 0x7f2e8c06e280 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ea013dd00 tx=0x7f2e94006bf0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.245 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 10 monclient: shutdown 2023-11-02T16:13:33.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:33.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 msgr2=0x7f2ea013c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2ea6192640 1 --2- 172.21.15.174:0/4148813429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea00b0a00 0x7f2ea013c400 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009e50 tx=0x7f2e90004a40 comp rx=0 tx=0).stop 2023-11-02T16:13:33.246 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.236+0000 7f2e9ffff640 1 -- 172.21.15.174:0/4148813429 reap_dead start 2023-11-02T16:13:33.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.237+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 shutdown_connections 2023-11-02T16:13:33.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.237+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 >> 172.21.15.174:0/4148813429 conn(0x7f2ea0065960 msgr2=0x7f2ea0073580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:33.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.237+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 shutdown_connections 2023-11-02T16:13:33.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.237+0000 7f2ea6192640 1 -- 172.21.15.174:0/4148813429 wait complete. 2023-11-02T16:13:33.247 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.237+0000 7f2ea6192640 1 librados: shutdown 2023-11-02T16:13:33.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-02T16:13:33.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.356+0000 7f3e954fb640 10 monclient: get_monmap_and_config 2023-11-02T16:13:33.358 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.356+0000 7f3e954fb640 10 monclient: build_initial_monmap 2023-11-02T16:13:33.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.356+0000 7f3e954fb640 10 monclient: monmap: 2023-11-02T16:13:33.359 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:33.359 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.359 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:33.357309+0000 2023-11-02T16:13:33.359 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:33.357309+0000 2023-11-02T16:13:33.360 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:33.360 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.360 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:33.360 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:33.361 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:33.361 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:33.361 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:33.361 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 Processor -- start 2023-11-02T16:13:33.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 -- start start 2023-11-02T16:13:33.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient: init 2023-11-02T16:13:33.363 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:33.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2023-11-02T16:13:33.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3e900b4570 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:33.364 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3e900b0ed0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:33.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e90144d40 0x7f3e9013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-a con 0x7f3e90144d40 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:33.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900a9780 con 0x7f3e900b4570 2023-11-02T16:13:33.365 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e901451d0 con 0x7f3e900b0ed0 2023-11-02T16:13:33.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900aa1d0 con 0x7f3e90144d40 2023-11-02T16:13:33.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:33.366 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.357+0000 7f3e954fb640 10 monclient(hunting): authenticate will time out at 3740.699348s 2023-11-02T16:13:33.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.359+0000 7f3e8e7fc640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3e900b0ed0 0x7f3e9013e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.174:51464/0 (socket says 172.21.15.174:51464) 2023-11-02T16:13:33.367 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.359+0000 7f3e8e7fc640 1 -- 172.21.15.174:0/894896637 learned_addr learned my addr 172.21.15.174:0/894896637 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:33.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962522521 0 0) 0x7f3e901451d0 con 0x7f3e900b0ed0 2023-11-02T16:13:33.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 10 monclient(hunting): _negotiate 2023-11-02T16:13:33.389 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:33.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 10 monclient(hunting): my global_id is 21592 2023-11-02T16:13:33.390 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.387+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e78003520 con 0x7f3e900b0ed0 2023-11-02T16:13:33.391 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.389+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/894896637 >> v2:172.21.15.174:3300/0 conn(0x7f3e90144d40 0x7f3e9013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490141162 0 0) 0x7f3e78003520 con 0x7f3e900b0ed0 2023-11-02T16:13:33.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 >> v2:172.21.15.174:3300/0 conn(0x7f3e90144d40 msgr2=0x7f3e9013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.394 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 --2- 172.21.15.174:0/894896637 >> v2:172.21.15.174:3300/0 conn(0x7f3e90144d40 0x7f3e9013ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 >> v1:172.21.15.174:6789/0 conn(0x7f3e900b4570 legacy=0x7f3e900a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _finish_hunting 0 2023-11-02T16:13:33.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 monclient: found mon.noname-c-legacy 2023-11-02T16:13:33.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:33.395 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:172.21.15.175:6789/0 2023-11-02T16:13:33.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900aa4b0 con 0x7f3e900b0ed0 2023-11-02T16:13:33.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _finish_auth 0 2023-11-02T16:13:33.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _check_auth_tickets 2023-11-02T16:13:33.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:33.396 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 692219148 0 0) 0x7f3e840016c0 con 0x7f3e900b0ed0 2023-11-02T16:13:33.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e954fb640 5 monclient: authenticate success, global_id 21592 2023-11-02T16:13:33.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e954fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:33.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2023-11-02T16:13:33.397 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: dump: 2023-11-02T16:13:33.398 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.398 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.398 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.398 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.398 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.399 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.399 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.399 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.399 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.399 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 monclient: mon.b has (v2) addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] but i'm connected to v1:172.21.15.175:6789/0, reconnecting 2023-11-02T16:13:33.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:33.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 >> v1:172.21.15.175:6789/0 conn(0x7f3e900b0ed0 legacy=0x7f3e9013e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.391+0000 7f3e8dffb640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T16:13:33.400 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780039c0 0x7f3e78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): picked mon.a con 0x7f3e780039c0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:33.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e780047c0 0x7f3e78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): picked mon.c con 0x7f3e780047c0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:33.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 0x7f3e7800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.401 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): picked mon.b con 0x7f3e780053d0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:33.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900aa1d0 con 0x7f3e780039c0 2023-11-02T16:13:33.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7800c310 con 0x7f3e780053d0 2023-11-02T16:13:33.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7800c4d0 con 0x7f3e780047c0 2023-11-02T16:13:33.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:33.403 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e8dffb640 10 monclient(hunting): _finish_auth 0 2023-11-02T16:13:33.404 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.392+0000 7f3e954fb640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:33.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.394+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 0x7f3e7800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.408 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): get_auth_request con 0x7f3e780053d0 auth_method 0 2023-11-02T16:13:33.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:33.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2023-11-02T16:13:33.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:33.409 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:33.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.395+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-11-02T16:13:33.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_done global_id 21592 payload 293 2023-11-02T16:13:33.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e780047c0 msgr2=0x7f3e78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.410 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e780047c0 0x7f3e78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780039c0 msgr2=0x7f3e78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:33.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780039c0 0x7f3e78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:33.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 monclient: found mon.b 2023-11-02T16:13:33.411 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:33.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:33.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/894896637 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78003520 con 0x7f3e780053d0 2023-11-02T16:13:33.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:33.412 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.397+0000 7f3e8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.398+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/894896637 reap_dead start 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.399+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 0x7f3e7800ba30 secure :-1 s=READY pgs=5282 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007f60 tx=0x7f3e780057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8dfb25f491c71f server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88033600 con 0x7f3e780053d0 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.413 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:33.414 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: dump: 2023-11-02T16:13:33.414 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.414 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.414 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.414 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.415 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.415 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.415 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.415 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.415 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.416 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.416 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8802f050 con 0x7f3e780053d0 2023-11-02T16:13:33.416 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e954fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:13:33.416 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:33.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e954fb640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:33.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 1 -- 172.21.15.174:0/894896637 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88033960 con 0x7f3e780053d0 2023-11-02T16:13:33.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e954fb640 4 set_mon_vals no callback set 2023-11-02T16:13:33.417 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:33.418 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.402+0000 7f3e8dffb640 10 monclient: dump: 2023-11-02T16:13:33.418 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.418 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.418 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.418 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.419 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.419 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.419 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.419 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.420 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.420 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 10 monclient: shutdown 2023-11-02T16:13:33.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:33.420 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 -- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 msgr2=0x7f3e7800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 0x7f3e7800ba30 secure :-1 s=READY pgs=5282 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007f60 tx=0x7f3e780057f0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 -- 172.21.15.174:0/894896637 shutdown_connections 2023-11-02T16:13:33.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 --2- 172.21.15.174:0/894896637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e780053d0 0x7f3e7800ba30 unknown :-1 s=CLOSED pgs=5282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 -- 172.21.15.174:0/894896637 >> 172.21.15.174:0/894896637 conn(0x7f3e90065d50 msgr2=0x7f3e90066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:33.421 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 -- 172.21.15.174:0/894896637 shutdown_connections 2023-11-02T16:13:33.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.403+0000 7f3e954fb640 1 -- 172.21.15.174:0/894896637 wait complete. 2023-11-02T16:13:33.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.404+0000 7f3e954fb640 10 monclient: build_initial_monmap 2023-11-02T16:13:33.422 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.404+0000 7f3e954fb640 10 monclient: monmap: 2023-11-02T16:13:33.422 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:33.422 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:33.423 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:33.423 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:33.423 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:33.423 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.423 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:33.424 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:33.424 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:33.424 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.404+0000 7f3e954fb640 1 librados: starting msgr at 2023-11-02T16:13:33.424 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.404+0000 7f3e954fb640 1 librados: starting objecter 2023-11-02T16:13:33.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 Processor -- start 2023-11-02T16:13:33.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 -- start start 2023-11-02T16:13:33.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 librados: setting wanted keys 2023-11-02T16:13:33.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 librados: calling monclient init 2023-11-02T16:13:33.425 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient: init 2023-11-02T16:13:33.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:33.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T16:13:33.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780053d0 0x7f3e900ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.426 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-a con 0x7f3e780053d0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:33.427 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900b0460 0x7f3e901d6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-c con 0x7f3e900b0460 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:33.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): picked mon.noname-b con 0x7f3e900b08c0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:33.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.428 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9013b760 con 0x7f3e780053d0 2023-11-02T16:13:33.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900adf20 con 0x7f3e900b08c0 2023-11-02T16:13:33.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:33.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901d6130 con 0x7f3e900b0460 2023-11-02T16:13:33.429 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:33.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.405+0000 7f3e954fb640 10 monclient(hunting): authenticate will time out at 3740.747523s 2023-11-02T16:13:33.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:54988/0 (socket says 172.21.15.174:54988) 2023-11-02T16:13:33.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/2584477689 learned_addr learned my addr 172.21.15.174:0/2584477689 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:33.430 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 10 monclient(hunting): get_auth_request con 0x7f3e900b08c0 auth_method 0 2023-11-02T16:13:33.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:33.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:33.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.406+0000 7f3e8f7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:33.431 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.407+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:33.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.407+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:33.432 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.407+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:33.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 10 monclient(hunting): handle_auth_done global_id 21595 payload 824 2023-11-02T16:13:33.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900b0460 msgr2=0x7f3e901d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900b0460 0x7f3e901d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.435 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780053d0 msgr2=0x7f3e900ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780053d0 0x7f3e900ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:33.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 monclient: found mon.noname-b 2023-11-02T16:13:33.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:33.436 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.409+0000 7f3e8f7fe640 1 -- 172.21.15.174:0/2584477689 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e901d6e80 con 0x7f3e900b08c0 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.411+0000 7f3e8f7fe640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 secure :-1 s=READY pgs=5283 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800c400 tx=0x7f3e88004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.411+0000 7f3e777fe640 10 client.?.objecter ms_handle_connect 0x7f3e900b08c0 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.411+0000 7f3e777fe640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88033ea0 con 0x7f3e900b08c0 2023-11-02T16:13:33.437 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:33.438 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: dump: 2023-11-02T16:13:33.438 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.438 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.439 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.439 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.439 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.439 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.439 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.440 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.440 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.440 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: _finish_auth 0 2023-11-02T16:13:33.440 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: _check_auth_tickets 2023-11-02T16:13:33.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:33.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8803c810 con 0x7f3e900b08c0 2023-11-02T16:13:33.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 5 monclient: authenticate success, global_id 21595 2023-11-02T16:13:33.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 10 monclient: _renew_subs 2023-11-02T16:13:33.441 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:33.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9013be90 con 0x7f3e900b08c0 2023-11-02T16:13:33.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:33.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8803cb80 con 0x7f3e900b08c0 2023-11-02T16:13:33.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:33.442 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:13:33.443 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e777fe640 10 monclient: dump: 2023-11-02T16:13:33.443 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:33.443 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.443 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.443 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:33.444 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:33.444 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.444 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:33.444 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:33.444 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:33.445 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:33.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e8d7fa640 4 set_mon_vals no callback set 2023-11-02T16:13:33.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 10 client.21595.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:33.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 10 monclient: _renew_subs 2023-11-02T16:13:33.445 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e900aece0 con 0x7f3e900b08c0 2023-11-02T16:13:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.415+0000 7f3e954fb640 1 librados: init done 2023-11-02T16:13:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.416+0000 7f3e954fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.416+0000 7f3e954fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.416+0000 7f3e954fb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:13:33.447 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.416+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9013c0b0 con 0x7f3e900b08c0 2023-11-02T16:13:33.655 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.652+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e8803a040 con 0x7f3e900b08c0 2023-11-02T16:13:33.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.653+0000 7f3e777fe640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 0x7f3e7c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:33.656 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.653+0000 7f3e8effd640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 0x7f3e7c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:33.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.653+0000 7f3e8effd640 10 monclient: get_auth_request con 0x7f3e7c06bdc0 auth_method 0 2023-11-02T16:13:33.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.654+0000 7f3e777fe640 10 client.21595.objecter ms_handle_connect 0x7f3e7c06bdc0 2023-11-02T16:13:33.657 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.654+0000 7f3e8effd640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 0x7f3e7c06e280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001550 tx=0x7f3e80007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:33.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.664+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f3e8807b010 con 0x7f3e900b08c0 2023-11-02T16:13:33.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.664+0000 7f3e777fe640 10 client.21595.objecter ms_dispatch 0x7f3e90000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:33.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.664+0000 7f3e777fe640 3 client.21595.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:33.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.664+0000 7f3e777fe640 3 client.21595.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:33.666 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.664+0000 7f3e777fe640 20 client.21595.objecter dump_active .. 0 homeless 2023-11-02T16:13:33.696 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.695+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3e88076c50 con 0x7f3e900b08c0 2023-11-02T16:13:33.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.695+0000 7f3e777fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:33.697 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.695+0000 7f3e777fe640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:33.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.873+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e901d62f0 con 0x7f3e7c06bdc0 2023-11-02T16:13:33.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.874+0000 7f3e777fe640 1 -- 172.21.15.174:0/2584477689 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3e901d62f0 con 0x7f3e7c06bdc0 2023-11-02T16:13:33.876 INFO:teuthology.orchestra.run.smithi174.stderr:instructed osd(s) 7 to deep-scrub 2023-11-02T16:13:33.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.875+0000 7f3e954fb640 10 librados: watch_flush enter 2023-11-02T16:13:33.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.875+0000 7f3e954fb640 10 librados: watch_flush exit 2023-11-02T16:13:33.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.875+0000 7f3e954fb640 20 client.21595.objecter shutdown clearing up homeless session... 2023-11-02T16:13:33.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.875+0000 7f3e954fb640 10 client.21595.objecter successfully canceled tick 2023-11-02T16:13:33.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 msgr2=0x7f3e7c06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 0x7f3e7c06e280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001550 tx=0x7f3e80007990 comp rx=0 tx=0).stop 2023-11-02T16:13:33.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 10 monclient: shutdown 2023-11-02T16:13:33.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:33.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 msgr2=0x7f3e900ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:33.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 secure :-1 s=READY pgs=5283 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800c400 tx=0x7f3e88004ab0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 shutdown_connections 2023-11-02T16:13:33.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e900b08c0 0x7f3e900ae5c0 unknown :-1 s=CLOSED pgs=5283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f3e7c06bdc0 0x7f3e7c06e280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900b0460 0x7f3e901d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 --2- 172.21.15.174:0/2584477689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e780053d0 0x7f3e900ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:33.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 >> 172.21.15.174:0/2584477689 conn(0x7f3e90065d50 msgr2=0x7f3e9013c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:33.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 shutdown_connections 2023-11-02T16:13:33.881 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 -- 172.21.15.174:0/2584477689 wait complete. 2023-11-02T16:13:33.882 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.876+0000 7f3e954fb640 1 librados: shutdown 2023-11-02T16:13:33.891 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:13:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.994+0000 7f2a681f0640 10 monclient: get_monmap_and_config 2023-11-02T16:13:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.994+0000 7f2a681f0640 10 monclient: build_initial_monmap 2023-11-02T16:13:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.994+0000 7f2a681f0640 10 monclient: monmap: 2023-11-02T16:13:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:33.995622+0000 2023-11-02T16:13:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:33.995622+0000 2023-11-02T16:13:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:33.999 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.995+0000 7f2a681f0640 1 Processor -- start 2023-11-02T16:13:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.995+0000 7f2a681f0640 1 -- start start 2023-11-02T16:13:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.995+0000 7f2a681f0640 10 monclient: init 2023-11-02T16:13:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:34.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2023-11-02T16:13:34.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a600b4260 0x7f2a600a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-a con 0x7f2a600b4260 addr v2:172.21.15.174:3300/0 2023-11-02T16:13:34.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2a60141730 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:34.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2a6013e520 0x7f2a6013e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-c con 0x7f2a6013e520 addr v2:172.21.15.175:3300/0 2023-11-02T16:13:34.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600a9210 con 0x7f2a60141730 2023-11-02T16:13:34.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:34.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600aa550 con 0x7f2a600b4260 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600b4640 con 0x7f2a6013e520 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.996+0000 7f2a681f0640 10 monclient(hunting): authenticate will time out at 3741.337807s 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.997+0000 7f2a65f8c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a600b4260 0x7f2a600a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:34.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a600b4260 0x7f2a600a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41022/0 (socket says 172.21.15.174:41022) 2023-11-02T16:13:34.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 1 -- 172.21.15.174:0/3317970147 learned_addr learned my addr 172.21.15.174:0/3317970147 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:34.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): get_auth_request con 0x7f2a600b4260 auth_method 0 2023-11-02T16:13:34.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:34.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:34.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:34.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:34.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:34.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:33.998+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:34.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_done global_id 15840 payload 274 2023-11-02T16:13:34.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 1 -- 172.21.15.174:0/3317970147 >> v2:172.21.15.175:3300/0 conn(0x7f2a6013e520 msgr2=0x7f2a6013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:34.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 1 --2- 172.21.15.174:0/3317970147 >> v2:172.21.15.175:3300/0 conn(0x7f2a6013e520 0x7f2a6013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 1 -- 172.21.15.174:0/3317970147 >> v1:172.21.15.174:6789/0 conn(0x7f2a60141730 legacy=0x7f2a60141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:13:34.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 10 monclient: _finish_hunting 0 2023-11-02T16:13:34.008 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 1 monclient: found mon.noname-a 2023-11-02T16:13:34.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:34.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:34.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.000+0000 7f2a65f8c640 1 -- 172.21.15.174:0/3317970147 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a601422b0 con 0x7f2a600b4260 2023-11-02T16:13:34.009 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.002+0000 7f2a65f8c640 1 --2- 172.21.15.174:0/3317970147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a600a8870 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098a0 tx=0x7f2a5c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4aefee1a73daf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:34.012 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.006+0000 7f2a64f8a640 1 -- 172.21.15.174:0/3317970147 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0317c0 con 0x7f2a600b4260 2023-11-02T16:13:34.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:34.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:34.013 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: dump: 2023-11-02T16:13:34.013 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:34.014 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:34.014 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.014 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.014 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:34.014 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:34.015 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:34.015 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:34.015 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:34.015 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: _finish_auth 0 2023-11-02T16:13:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: _check_auth_tickets 2023-11-02T16:13:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 1 -- 172.21.15.174:0/3317970147 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c031920 con 0x7f2a600b4260 2023-11-02T16:13:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:34.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 1 -- 172.21.15.174:0/3317970147 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c031cb0 con 0x7f2a600b4260 2023-11-02T16:13:34.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 5 monclient: authenticate success, global_id 15840 2023-11-02T16:13:34.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:34.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 4 set_mon_vals no callback set 2023-11-02T16:13:34.017 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:34.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:34.018 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a64f8a640 10 monclient: dump: 2023-11-02T16:13:34.018 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:34.018 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:34.018 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.019 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.019 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:34.019 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:34.019 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:34.020 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:34.020 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:34.020 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 10 monclient: shutdown 2023-11-02T16:13:34.020 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:34.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 -- 172.21.15.174:0/3317970147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 msgr2=0x7f2a600a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:34.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 --2- 172.21.15.174:0/3317970147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a600a8870 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098a0 tx=0x7f2a5c02b860 comp rx=0 tx=0).stop 2023-11-02T16:13:34.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 -- 172.21.15.174:0/3317970147 shutdown_connections 2023-11-02T16:13:34.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 --2- 172.21.15.174:0/3317970147 >> v2:172.21.15.175:3300/0 conn(0x7f2a6013e520 0x7f2a6013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.021 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 --2- 172.21.15.174:0/3317970147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a600a8870 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.022 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 -- 172.21.15.174:0/3317970147 >> 172.21.15.174:0/3317970147 conn(0x7f2a60065d50 msgr2=0x7f2a60066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:34.022 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 -- 172.21.15.174:0/3317970147 shutdown_connections 2023-11-02T16:13:34.022 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.007+0000 7f2a681f0640 1 -- 172.21.15.174:0/3317970147 wait complete. 2023-11-02T16:13:34.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 10 monclient: build_initial_monmap 2023-11-02T16:13:34.023 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 10 monclient: monmap: 2023-11-02T16:13:34.023 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:34.023 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:34.023 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:34.025 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:34.025 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 librados: starting msgr at 2023-11-02T16:13:34.025 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 librados: starting objecter 2023-11-02T16:13:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 Processor -- start 2023-11-02T16:13:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 -- start start 2023-11-02T16:13:34.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 librados: setting wanted keys 2023-11-02T16:13:34.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 1 librados: calling monclient init 2023-11-02T16:13:34.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.008+0000 7f2a681f0640 10 monclient: init 2023-11-02T16:13:34.028 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:34.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient: _add_conns ranks=[0,2,1] 2023-11-02T16:13:34.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-a con 0x7f2a600b4260 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:34.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6013e520 0x7f2a6013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.029 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-c con 0x7f2a6013e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:34.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60141730 0x7f2a601404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): picked mon.noname-b con 0x7f2a60141730 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:34.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:34.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013b230 con 0x7f2a600b4260 2023-11-02T16:13:34.030 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:34.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013e100 con 0x7f2a60141730 2023-11-02T16:13:34.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:34.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013e2c0 con 0x7f2a6013e520 2023-11-02T16:13:34.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:34.031 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.009+0000 7f2a681f0640 10 monclient(hunting): authenticate will time out at 3741.351249s 2023-11-02T16:13:34.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:34.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41034/0 (socket says 172.21.15.174:41034) 2023-11-02T16:13:34.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 1 -- 172.21.15.174:0/1386798168 learned_addr learned my addr 172.21.15.174:0/1386798168 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:34.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): get_auth_request con 0x7f2a600b4260 auth_method 0 2023-11-02T16:13:34.032 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:34.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:34.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:34.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:34.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:34.033 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.010+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:34.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 10 monclient(hunting): handle_auth_done global_id 15843 payload 824 2023-11-02T16:13:34.040 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 -- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6013e520 msgr2=0x7f2a6013ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:34.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6013e520 0x7f2a6013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 -- 172.21.15.174:0/1386798168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60141730 msgr2=0x7f2a601404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:34.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60141730 0x7f2a601404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 10 monclient: _finish_hunting 0 2023-11-02T16:13:34.041 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 monclient: found mon.noname-a 2023-11-02T16:13:34.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:34.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:34.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.013+0000 7f2a65f8c640 1 -- 172.21.15.174:0/1386798168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0094d0 con 0x7f2a600b4260 2023-11-02T16:13:34.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.016+0000 7f2a65f8c640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c000c00 tx=0x7f2a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:34.042 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.016+0000 7f2a567fc640 10 client.?.objecter ms_handle_connect 0x7f2a600b4260 2023-11-02T16:13:34.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.016+0000 7f2a567fc640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:34.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c032860 con 0x7f2a600b4260 2023-11-02T16:13:34.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:34.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:34.043 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: dump: 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:34.044 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:34.045 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:34.045 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:34.045 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:34.045 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.045 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:34.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:34.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:34.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c0329c0 con 0x7f2a600b4260 2023-11-02T16:13:34.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 5 monclient: authenticate success, global_id 15843 2023-11-02T16:13:34.046 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 10 monclient: _renew_subs 2023-11-02T16:13:34.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:34.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6013aca0 con 0x7f2a600b4260 2023-11-02T16:13:34.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:34.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c032d50 con 0x7f2a600b4260 2023-11-02T16:13:34.047 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:34.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a57fff640 4 set_mon_vals no callback set 2023-11-02T16:13:34.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 10 client.15843.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:34.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:34.048 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a567fc640 10 monclient: dump: 2023-11-02T16:13:34.048 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:34.049 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:34.050 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:34.050 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:34.050 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:34.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 10 monclient: _renew_subs 2023-11-02T16:13:34.050 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:34.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a601d6bf0 con 0x7f2a600b4260 2023-11-02T16:13:34.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.021+0000 7f2a681f0640 1 librados: init done 2023-11-02T16:13:34.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.022+0000 7f2a681f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:34.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.022+0000 7f2a681f0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:34.051 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.022+0000 7f2a681f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:34.052 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.022+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6013af90 con 0x7f2a600b4260 2023-11-02T16:13:34.072 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.071+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2a5c04e450 con 0x7f2a600b4260 2023-11-02T16:13:34.073 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.071+0000 7f2a567fc640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 0x7f2a4406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:34.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.072+0000 7f2a6578b640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 0x7f2a4406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:34.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.072+0000 7f2a6578b640 10 monclient: get_auth_request con 0x7f2a4406bdc0 auth_method 0 2023-11-02T16:13:34.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.072+0000 7f2a6578b640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 0x7f2a4406e280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2a6013b9e0 tx=0x7f2a50006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:34.074 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.072+0000 7f2a567fc640 10 client.15843.objecter ms_handle_connect 0x7f2a4406bdc0 2023-11-02T16:13:34.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f2a5c07a2f0 con 0x7f2a600b4260 2023-11-02T16:13:34.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 10 client.15843.objecter ms_dispatch 0x7f2a60000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:34.339 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 3 client.15843.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:34.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 3 client.15843.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:34.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 20 client.15843.objecter dump_active .. 0 homeless 2023-11-02T16:13:34.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a5c076630 con 0x7f2a600b4260 2023-11-02T16:13:34.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:34.340 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.337+0000 7f2a567fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:34.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.521+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a600aa790 con 0x7f2a4406bdc0 2023-11-02T16:13:34.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.526+0000 7f2a567fc640 1 -- 172.21.15.174:0/1386798168 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+150880 (secure 0 0 0) 0x7f2a50012390 con 0x7f2a4406bdc0 2023-11-02T16:13:34.529 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:34.530 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:13:34.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 10 librados: watch_flush enter 2023-11-02T16:13:34.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 10 librados: watch_flush exit 2023-11-02T16:13:34.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 20 client.15843.objecter shutdown clearing up homeless session... 2023-11-02T16:13:34.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 10 client.15843.objecter successfully canceled tick 2023-11-02T16:13:34.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 msgr2=0x7f2a4406e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:34.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 0x7f2a4406e280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2a6013b9e0 tx=0x7f2a50006ba0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 10 monclient: shutdown 2023-11-02T16:13:34.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:34.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 msgr2=0x7f2a6013f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:34.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c000c00 tx=0x7f2a5c0046c0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 shutdown_connections 2023-11-02T16:13:34.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60141730 0x7f2a601404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f2a4406bdc0 0x7f2a4406e280 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6013e520 0x7f2a6013ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 --2- 172.21.15.174:0/1386798168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a600b4260 0x7f2a6013f7a0 unknown :-1 s=CLOSED pgs=3202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:34.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 >> 172.21.15.174:0/1386798168 conn(0x7f2a60065d50 msgr2=0x7f2a60066130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:34.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 shutdown_connections 2023-11-02T16:13:34.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 -- 172.21.15.174:0/1386798168 wait complete. 2023-11-02T16:13:34.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:34.528+0000 7f2a681f0640 1 librados: shutdown 2023-11-02T16:13:34.541 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2224,"stamp":"2023-11-02T16:13:33.406473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609908,"kb_used_data":86068,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364812,"statfs":{"total":773094113280,"available":772469567488,"internally_reserved":0,"allocated":88133632,"data_stored":51451729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002584"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.936502+0000","last_change":"2023-11-02T16:13:17.157671+0000","last_active":"2023-11-02T16:13:24.936502+0000","last_peered":"2023-11-02T16:13:24.936502+0000","last_clean":"2023-11-02T16:13:24.936502+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:24.936502+0000","last_undegraded":"2023-11-02T16:13:24.936502+0000","last_fullsized":"2023-11-02T16:13:24.936502+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1013,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.544260+0000","last_change":"2023-11-02T16:07:27.307241+0000","last_active":"2023-11-02T16:13:24.544260+0000","last_peered":"2023-11-02T16:13:24.544260+0000","last_clean":"2023-11-02T16:13:24.544260+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:24.544260+0000","last_undegraded":"2023-11-02T16:13:24.544260+0000","last_fullsized":"2023-11-02T16:13:24.544260+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":935,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.756848+0000","last_change":"2023-11-02T16:07:27.395825+0000","last_active":"2023-11-02T16:13:22.756848+0000","last_peered":"2023-11-02T16:13:22.756848+0000","last_clean":"2023-11-02T16:13:22.756848+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:22.756848+0000","last_undegraded":"2023-11-02T16:13:22.756848+0000","last_fullsized":"2023-11-02T16:13:22.756848+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:25:43.494367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":975,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.387123+0000","last_change":"2023-11-02T15:35:17.345372+0000","last_active":"2023-11-02T16:13:22.387123+0000","last_peered":"2023-11-02T16:13:22.387123+0000","last_clean":"2023-11-02T16:13:22.387123+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:22.387123+0000","last_undegraded":"2023-11-02T16:13:22.387123+0000","last_fullsized":"2023-11-02T16:13:22.387123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:39:06.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":975,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:25.294292+0000","last_change":"2023-11-02T15:35:14.211126+0000","last_active":"2023-11-02T16:13:25.294292+0000","last_peered":"2023-11-02T16:13:25.294292+0000","last_clean":"2023-11-02T16:13:25.294292+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:25.294292+0000","last_undegraded":"2023-11-02T16:13:25.294292+0000","last_fullsized":"2023-11-02T16:13:25.294292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":922,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.753448+0000","last_change":"2023-11-02T15:59:17.698985+0000","last_active":"2023-11-02T16:13:22.753448+0000","last_peered":"2023-11-02T16:13:22.753448+0000","last_clean":"2023-11-02T16:13:22.753448+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:22.753448+0000","last_undegraded":"2023-11-02T16:13:22.753448+0000","last_fullsized":"2023-11-02T16:13:22.753448+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:31:01.071757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1001,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:25.281433+0000","last_change":"2023-11-02T16:13:16.820908+0000","last_active":"2023-11-02T16:13:25.281433+0000","last_peered":"2023-11-02T16:13:25.281433+0000","last_clean":"2023-11-02T16:13:25.281433+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:25.281433+0000","last_undegraded":"2023-11-02T16:13:25.281433+0000","last_fullsized":"2023-11-02T16:13:25.281433+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":975,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:25.288407+0000","last_change":"2023-11-02T15:35:30.105402+0000","last_active":"2023-11-02T16:13:25.288407+0000","last_peered":"2023-11-02T16:13:25.288407+0000","last_clean":"2023-11-02T16:13:25.288407+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:25.288407+0000","last_undegraded":"2023-11-02T16:13:25.288407+0000","last_fullsized":"2023-11-02T16:13:25.288407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":964,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:27.713111+0000","last_change":"2023-11-02T16:07:27.407922+0000","last_active":"2023-11-02T16:13:27.713111+0000","last_peered":"2023-11-02T16:13:27.713111+0000","last_clean":"2023-11-02T16:13:27.713111+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:27.713111+0000","last_undegraded":"2023-11-02T16:13:27.713111+0000","last_fullsized":"2023-11-02T16:13:27.713111+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1004,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.554873+0000","last_change":"2023-11-02T16:13:16.885325+0000","last_active":"2023-11-02T16:13:24.554873+0000","last_peered":"2023-11-02T16:13:24.554873+0000","last_clean":"2023-11-02T16:13:24.554873+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:24.554873+0000","last_undegraded":"2023-11-02T16:13:24.554873+0000","last_fullsized":"2023-11-02T16:13:24.554873+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1014,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.541860+0000","last_change":"2023-11-02T16:13:16.913467+0000","last_active":"2023-11-02T16:13:24.541860+0000","last_peered":"2023-11-02T16:13:24.541860+0000","last_clean":"2023-11-02T16:13:24.541860+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:24.541860+0000","last_undegraded":"2023-11-02T16:13:24.541860+0000","last_fullsized":"2023-11-02T16:13:24.541860+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1013,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.537110+0000","last_change":"2023-11-02T16:13:17.189587+0000","last_active":"2023-11-02T16:13:24.537110+0000","last_peered":"2023-11-02T16:13:24.537110+0000","last_clean":"2023-11-02T16:13:24.537110+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:24.537110+0000","last_undegraded":"2023-11-02T16:13:24.537110+0000","last_fullsized":"2023-11-02T16:13:24.537110+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":988,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.541123+0000","last_change":"2023-11-02T15:59:17.640186+0000","last_active":"2023-11-02T16:13:24.541123+0000","last_peered":"2023-11-02T16:13:24.541123+0000","last_clean":"2023-11-02T16:13:24.541123+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:24.541123+0000","last_undegraded":"2023-11-02T16:13:24.541123+0000","last_fullsized":"2023-11-02T16:13:24.541123+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":992,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.539490+0000","last_change":"2023-11-02T15:59:17.577430+0000","last_active":"2023-11-02T16:13:24.539490+0000","last_peered":"2023-11-02T16:13:24.539490+0000","last_clean":"2023-11-02T16:13:24.539490+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:24.539490+0000","last_undegraded":"2023-11-02T16:13:24.539490+0000","last_fullsized":"2023-11-02T16:13:24.539490+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":979,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:27.087117+0000","last_change":"2023-11-02T15:35:23.837202+0000","last_active":"2023-11-02T16:13:27.087117+0000","last_peered":"2023-11-02T16:13:27.087117+0000","last_clean":"2023-11-02T16:13:27.087117+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:27.087117+0000","last_undegraded":"2023-11-02T16:13:27.087117+0000","last_fullsized":"2023-11-02T16:13:27.087117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":981,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.381965+0000","last_change":"2023-11-02T15:34:02.025414+0000","last_active":"2023-11-02T16:13:22.381965+0000","last_peered":"2023-11-02T16:13:22.381965+0000","last_clean":"2023-11-02T16:13:22.381965+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:22.381965+0000","last_undegraded":"2023-11-02T16:13:22.381965+0000","last_fullsized":"2023-11-02T16:13:22.381965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:21:25.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":993,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:27.085038+0000","last_change":"2023-11-02T15:59:17.883690+0000","last_active":"2023-11-02T16:13:27.085038+0000","last_peered":"2023-11-02T16:13:27.085038+0000","last_clean":"2023-11-02T16:13:27.085038+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:27.085038+0000","last_undegraded":"2023-11-02T16:13:27.085038+0000","last_fullsized":"2023-11-02T16:13:27.085038+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":994,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.535545+0000","last_change":"2023-11-02T15:59:17.612075+0000","last_active":"2023-11-02T16:13:24.535545+0000","last_peered":"2023-11-02T16:13:24.535545+0000","last_clean":"2023-11-02T16:13:24.535545+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:24.535545+0000","last_undegraded":"2023-11-02T16:13:24.535545+0000","last_fullsized":"2023-11-02T16:13:24.535545+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1017,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:27.750644+0000","last_change":"2023-11-02T16:13:17.151154+0000","last_active":"2023-11-02T16:13:27.750644+0000","last_peered":"2023-11-02T16:13:27.750644+0000","last_clean":"2023-11-02T16:13:27.750644+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:27.750644+0000","last_undegraded":"2023-11-02T16:13:27.750644+0000","last_fullsized":"2023-11-02T16:13:27.750644+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":955,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:26.660291+0000","last_change":"2023-11-02T15:43:56.978148+0000","last_active":"2023-11-02T16:13:26.660291+0000","last_peered":"2023-11-02T16:13:26.660291+0000","last_clean":"2023-11-02T16:13:26.660291+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:26.660291+0000","last_undegraded":"2023-11-02T16:13:26.660291+0000","last_fullsized":"2023-11-02T16:13:26.660291+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":534,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.382202+0000","last_change":"2023-11-02T16:07:27.487367+0000","last_active":"2023-11-02T16:13:22.382202+0000","last_peered":"2023-11-02T16:13:22.382202+0000","last_clean":"2023-11-02T16:13:22.382202+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:22.382202+0000","last_undegraded":"2023-11-02T16:13:22.382202+0000","last_fullsized":"2023-11-02T16:13:22.382202+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:15:14.310785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1009,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.518004+0000","last_change":"2023-11-02T15:59:17.535951+0000","last_active":"2023-11-02T16:13:24.518004+0000","last_peered":"2023-11-02T16:13:24.518004+0000","last_clean":"2023-11-02T16:13:24.518004+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:24.518004+0000","last_undegraded":"2023-11-02T16:13:24.518004+0000","last_fullsized":"2023-11-02T16:13:24.518004+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":980,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.932678+0000","last_change":"2023-11-02T15:35:16.434956+0000","last_active":"2023-11-02T16:13:24.932678+0000","last_peered":"2023-11-02T16:13:24.932678+0000","last_clean":"2023-11-02T16:13:24.932678+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:24.932678+0000","last_undegraded":"2023-11-02T16:13:24.932678+0000","last_fullsized":"2023-11-02T16:13:24.932678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":990,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.371512+0000","last_change":"2023-11-02T16:07:27.307462+0000","last_active":"2023-11-02T16:13:22.371512+0000","last_peered":"2023-11-02T16:13:22.371512+0000","last_clean":"2023-11-02T16:13:22.371512+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:22.371512+0000","last_undegraded":"2023-11-02T16:13:22.371512+0000","last_fullsized":"2023-11-02T16:13:22.371512+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:53:09.965685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":938,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:26.644606+0000","last_change":"2023-11-02T15:35:10.705051+0000","last_active":"2023-11-02T16:13:26.644606+0000","last_peered":"2023-11-02T16:13:26.644606+0000","last_clean":"2023-11-02T16:13:26.644606+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:26.644606+0000","last_undegraded":"2023-11-02T16:13:26.644606+0000","last_fullsized":"2023-11-02T16:13:26.644606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":995,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.520554+0000","last_change":"2023-11-02T15:59:17.407544+0000","last_active":"2023-11-02T16:13:24.520554+0000","last_peered":"2023-11-02T16:13:24.520554+0000","last_clean":"2023-11-02T16:13:24.520554+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:24.520554+0000","last_undegraded":"2023-11-02T16:13:24.520554+0000","last_fullsized":"2023-11-02T16:13:24.520554+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":976,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:25.296841+0000","last_change":"2023-11-02T15:35:30.088014+0000","last_active":"2023-11-02T16:13:25.296841+0000","last_peered":"2023-11-02T16:13:25.296841+0000","last_clean":"2023-11-02T16:13:25.296841+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:25.296841+0000","last_undegraded":"2023-11-02T16:13:25.296841+0000","last_fullsized":"2023-11-02T16:13:25.296841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1020,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.527411+0000","last_change":"2023-11-02T16:13:17.216261+0000","last_active":"2023-11-02T16:13:24.527411+0000","last_peered":"2023-11-02T16:13:24.527411+0000","last_clean":"2023-11-02T16:13:24.527411+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:24.527411+0000","last_undegraded":"2023-11-02T16:13:24.527411+0000","last_fullsized":"2023-11-02T16:13:24.527411+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:34:00.052706+0000","last_clean_scrub_stamp":"2023-11-02T15:34:00.052706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1030'78","reported_seq":1101,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:30.530233+0000","last_change":"2023-11-02T16:13:30.536191+0000","last_active":"2023-11-02T16:13:30.530233+0000","last_peered":"2023-11-02T16:13:30.530233+0000","last_clean":"2023-11-02T16:13:30.530233+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:30.530233+0000","last_undegraded":"2023-11-02T16:13:30.530233+0000","last_fullsized":"2023-11-02T16:13:30.530233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'78","last_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_deep_scrub":"1030'78","last_deep_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_clean_scrub_stamp":"2023-11-02T16:13:30.528744+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:03:01.384002+0000","scrub_duration":0.116374897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"1030'8","reported_seq":1032,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:29.185182+0000","last_change":"2023-11-02T16:13:29.187351+0000","last_active":"2023-11-02T16:13:29.185182+0000","last_peered":"2023-11-02T16:13:29.185182+0000","last_clean":"2023-11-02T16:13:29.185182+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:29.185182+0000","last_undegraded":"2023-11-02T16:13:29.185182+0000","last_fullsized":"2023-11-02T16:13:29.185182+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'8","last_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_deep_scrub":"1030'8","last_deep_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_clean_scrub_stamp":"2023-11-02T16:13:29.183095+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:20:07.879623+0000","scrub_duration":0.12634673699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1005,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.522297+0000","last_change":"2023-11-02T16:13:17.007381+0000","last_active":"2023-11-02T16:13:24.522297+0000","last_peered":"2023-11-02T16:13:24.522297+0000","last_clean":"2023-11-02T16:13:24.522297+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:24.522297+0000","last_undegraded":"2023-11-02T16:13:24.522297+0000","last_fullsized":"2023-11-02T16:13:24.522297+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1005,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.374124+0000","last_change":"2023-11-02T16:13:16.951026+0000","last_active":"2023-11-02T16:13:22.374124+0000","last_peered":"2023-11-02T16:13:22.374124+0000","last_clean":"2023-11-02T16:13:22.374124+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:22.374124+0000","last_undegraded":"2023-11-02T16:13:22.374124+0000","last_fullsized":"2023-11-02T16:13:22.374124+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:01:56.841900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":965,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:26.654342+0000","last_change":"2023-11-02T16:13:17.159877+0000","last_active":"2023-11-02T16:13:26.654342+0000","last_peered":"2023-11-02T16:13:26.654342+0000","last_clean":"2023-11-02T16:13:26.654342+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:26.654342+0000","last_undegraded":"2023-11-02T16:13:26.654342+0000","last_fullsized":"2023-11-02T16:13:26.654342+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":990,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.385612+0000","last_change":"2023-11-02T15:43:57.129812+0000","last_active":"2023-11-02T16:13:22.385612+0000","last_peered":"2023-11-02T16:13:22.385612+0000","last_clean":"2023-11-02T16:13:22.385612+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:22.385612+0000","last_undegraded":"2023-11-02T16:13:22.385612+0000","last_fullsized":"2023-11-02T16:13:22.385612+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T15:53:42.294660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":979,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.370292+0000","last_change":"2023-11-02T15:35:17.386786+0000","last_active":"2023-11-02T16:13:22.370292+0000","last_peered":"2023-11-02T16:13:22.370292+0000","last_clean":"2023-11-02T16:13:22.370292+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:22.370292+0000","last_undegraded":"2023-11-02T16:13:22.370292+0000","last_fullsized":"2023-11-02T16:13:22.370292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:58:39.849736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":989,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.542700+0000","last_change":"2023-11-02T15:59:17.643298+0000","last_active":"2023-11-02T16:13:24.542700+0000","last_peered":"2023-11-02T16:13:24.542700+0000","last_clean":"2023-11-02T16:13:24.542700+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:24.542700+0000","last_undegraded":"2023-11-02T16:13:24.542700+0000","last_fullsized":"2023-11-02T16:13:24.542700+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":990,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.546041+0000","last_change":"2023-11-02T15:59:17.471575+0000","last_active":"2023-11-02T16:13:24.546041+0000","last_peered":"2023-11-02T16:13:24.546041+0000","last_clean":"2023-11-02T16:13:24.546041+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:24.546041+0000","last_undegraded":"2023-11-02T16:13:24.546041+0000","last_fullsized":"2023-11-02T16:13:24.546041+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":935,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.748913+0000","last_change":"2023-11-02T16:13:16.821060+0000","last_active":"2023-11-02T16:13:22.748913+0000","last_peered":"2023-11-02T16:13:22.748913+0000","last_clean":"2023-11-02T16:13:22.748913+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:22.748913+0000","last_undegraded":"2023-11-02T16:13:22.748913+0000","last_fullsized":"2023-11-02T16:13:22.748913+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:46.475704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1003,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.355338+0000","last_change":"2023-11-02T16:07:27.399974+0000","last_active":"2023-11-02T16:13:22.355338+0000","last_peered":"2023-11-02T16:13:22.355338+0000","last_clean":"2023-11-02T16:13:22.355338+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:22.355338+0000","last_undegraded":"2023-11-02T16:13:22.355338+0000","last_fullsized":"2023-11-02T16:13:22.355338+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:02:28.668857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1003,"reported_epoch":1029,"state":"active+clean","last_fresh":"2023-11-02T16:13:22.372530+0000","last_change":"2023-11-02T16:13:16.955603+0000","last_active":"2023-11-02T16:13:22.372530+0000","last_peered":"2023-11-02T16:13:22.372530+0000","last_clean":"2023-11-02T16:13:22.372530+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:22.372530+0000","last_undegraded":"2023-11-02T16:13:22.372530+0000","last_fullsized":"2023-11-02T16:13:22.372530+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:12:56.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1000,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:25.291326+0000","last_change":"2023-11-02T16:13:16.834330+0000","last_active":"2023-11-02T16:13:25.291326+0000","last_peered":"2023-11-02T16:13:25.291326+0000","last_clean":"2023-11-02T16:13:25.291326+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:25.291326+0000","last_undegraded":"2023-11-02T16:13:25.291326+0000","last_fullsized":"2023-11-02T16:13:25.291326+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544253,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6332119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":6.1360000000000001},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":9.6159999999999997}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":19.917000000000002},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":15.973000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":17.399999999999999},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":11.866}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":13.976000000000001},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":17.984999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":11.278},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":15.029999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":20.946000000000002},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":15.43}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":22.440999999999999},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":18.856999999999999}]}]},{"osd":1,"up_from":16,"seq":68719477189,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104372,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267468,"statfs":{"total":96636764160,"available":96529887232,"internally_reserved":0,"allocated":11317248,"data_stored":6729954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":8.7509999999999994,"5min":11.885,"15min":17.582000000000001},"min":{"1min":4.5650000000000004,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":13.523999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":9.3870000000000005},{"interface":"front","average":{"1min":8.1389999999999993,"5min":12.238,"15min":18.489999999999998},"min":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":11.058999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":12.643000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":10.146000000000001},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":11.872}]},{"osd":3,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":11.907,"5min":13.795999999999999,"15min":21.059999999999999},"min":{"1min":8.5370000000000008,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":26.161000000000001,"5min":77.778000000000006,"15min":549.92200000000003},"last":17.149999999999999},{"interface":"front","average":{"1min":12.789999999999999,"5min":17.585999999999999,"15min":24.571000000000002},"min":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":7.8280000000000003},"max":{"1min":26.827000000000002,"5min":108.244,"15min":542.60500000000002},"last":13.759}]},{"osd":4,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":107.46299999999999,"5min":110.161,"15min":49.548000000000002},"min":{"1min":7.8819999999999997,"5min":7.8819999999999997,"15min":6.1360000000000001},"max":{"1min":1884.9880000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":14.653},{"interface":"front","average":{"1min":111.589,"5min":110.36199999999999,"15min":51.369},"min":{"1min":7.6950000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"max":{"1min":1970.0229999999999,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":20.852}]},{"osd":5,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":16.436,"15min":25.068999999999999},"min":{"1min":8.0990000000000002,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":16.541,"5min":71.225999999999999,"15min":529.93499999999995},"last":10.670999999999999},{"interface":"front","average":{"1min":15.509,"5min":19.974,"15min":26.859000000000002},"min":{"1min":9.5839999999999996,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":23.559999999999999,"5min":122.14,"15min":555.77700000000004},"last":17.713999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":15.324999999999999,"5min":22.655000000000001,"15min":1553.2819999999999},"min":{"1min":9.0719999999999992,"5min":8.6210000000000004,"15min":8.1400000000000006},"max":{"1min":31.744,"5min":245.36500000000001,"15min":73455.679000000004},"last":11.329000000000001},{"interface":"front","average":{"1min":15.226000000000001,"5min":17.460999999999999,"15min":1549.6890000000001},"min":{"1min":9.5269999999999992,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":30.879999999999999,"5min":124.36499999999999,"15min":73453.190000000002},"last":21.965}]},{"osd":7,"last update":"Thu Nov 2 16:12:35 2023","interfaces":[{"interface":"back","average":{"1min":15.096,"5min":18.516999999999999,"15min":21.047000000000001},"min":{"1min":10.757999999999999,"5min":10.097,"15min":10.097},"max":{"1min":21.617999999999999,"5min":91.019999999999996,"15min":91.019999999999996},"last":19.940000000000001},{"interface":"front","average":{"1min":16.209,"5min":22.247,"15min":24.065999999999999},"min":{"1min":10.824999999999999,"5min":10.113,"15min":10.113},"max":{"1min":28.861999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":18.291}]}]},{"osd":0,"up_from":16,"seq":68719477213,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81900,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289940,"statfs":{"total":96636764160,"available":96552898560,"internally_reserved":0,"allocated":10915840,"data_stored":6331258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:12:55 2023","interfaces":[{"interface":"back","average":{"1min":8.7309999999999999,"5min":13.488,"15min":77.375},"min":{"1min":5.3490000000000002,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.488,"5min":107.59999999999999,"15min":5781.2129999999997},"last":6.29},{"interface":"front","average":{"1min":8.4719999999999995,"5min":11.718,"15min":74.570999999999998},"min":{"1min":6.2249999999999996,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":13.672000000000001,"5min":73.094999999999999,"15min":5443.4889999999996},"last":7.8810000000000002}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.023999999999999},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":9.2170000000000005}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":12.1,"5min":17.452999999999999,"15min":17.625},"min":{"1min":8.3399999999999999,"5min":7.46,"15min":7.46},"max":{"1min":29.192,"5min":156.99100000000001,"15min":234.29300000000001},"last":11.643000000000001},{"interface":"front","average":{"1min":12.449999999999999,"5min":17.09,"15min":15.265000000000001},"min":{"1min":7.6239999999999997,"5min":7.4950000000000001,"15min":6.6070000000000002},"max":{"1min":34.267000000000003,"5min":151.30099999999999,"15min":151.30099999999999},"last":8.5389999999999997}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":21213.947,"5min":17485.800999999999,"15min":5840.5320000000002},"min":{"1min":9.7330000000000005,"5min":7.1600000000000001,"15min":6.3200000000000003},"max":{"1min":64456.964,"5min":66161.077000000005,"15min":66161.077000000005},"last":14.416},{"interface":"front","average":{"1min":21222.184000000001,"5min":17490.220000000001,"15min":5843.1549999999997},"min":{"1min":11.762,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":64460.497000000003,"5min":66163.926000000007,"15min":66163.926000000007},"last":16.495999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":17.084},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":18.852}]},{"osd":6,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":18.556999999999999,"5min":22.667000000000002,"15min":1772.4739999999999},"min":{"1min":14.353999999999999,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":38.277000000000001,"5min":107.51000000000001,"15min":71055.725999999995},"last":20.728000000000002},{"interface":"front","average":{"1min":13.893000000000001,"5min":19.393000000000001,"15min":1770.963},"min":{"1min":7.7320000000000002,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":32.561,"5min":156.91,"15min":71055.100000000006},"last":15.134}]},{"osd":7,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19.140000000000001,"5min":25.041,"15min":30.241},"min":{"1min":13.130000000000001,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":44.636000000000003,"5min":205.88999999999999,"15min":205.88999999999999},"last":19.701000000000001},{"interface":"front","average":{"1min":11.23,"5min":17.503,"15min":22.859000000000002},"min":{"1min":8.0190000000000001,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":31.449999999999999,"5min":155.96600000000001,"15min":155.96600000000001},"last":13.087999999999999}]}]},{"osd":6,"up_from":387,"seq":1662152343956,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.405000000000001,"5min":14.585000000000001,"15min":20.376999999999999},"min":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.218},"max":{"1min":208.60499999999999,"5min":208.60499999999999,"15min":572.92100000000005},"last":4.508},{"interface":"front","average":{"1min":19.026,"5min":14.109999999999999,"15min":21.210000000000001},"min":{"1min":6.8200000000000003,"5min":4.6980000000000004,"15min":4.0419999999999998},"max":{"1min":172.51499999999999,"5min":172.51499999999999,"15min":630.37900000000002},"last":7.4669999999999996}]},{"osd":1,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17.109000000000002,"5min":16.506,"15min":1146.9380000000001},"min":{"1min":7.5700000000000003,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":77.540000000000006,"5min":143.55699999999999,"15min":41558.328999999998},"last":15.117000000000001},{"interface":"front","average":{"1min":21.494,"5min":17.533999999999999,"15min":1148.0730000000001},"min":{"1min":7.633,"5min":6.8019999999999996,"15min":5.8220000000000001},"max":{"1min":176.839,"5min":176.839,"15min":41560.286999999997},"last":9.0739999999999998}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":12.227},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":16.068999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":25.748000000000001,"5min":22.646000000000001,"15min":24.829000000000001},"min":{"1min":9.4220000000000006,"5min":8.5069999999999997,"15min":6.4480000000000004},"max":{"1min":260.11700000000002,"5min":260.11700000000002,"15min":633.899},"last":16.776},{"interface":"front","average":{"1min":20.872,"5min":19.306999999999999,"15min":25.396999999999998},"min":{"1min":8.0060000000000002,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":177.78100000000001,"5min":188.101,"15min":582.32399999999996},"last":11.662000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":17750.601999999999,"5min":4460.9930000000004,"15min":1504.991},"min":{"1min":10.298,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":66604.820000000007,"5min":67109.214999999997,"15min":67109.214999999997},"last":14.382999999999999},{"interface":"front","average":{"1min":17580.571,"5min":4423.1670000000004,"15min":1494.211},"min":{"1min":8.0969999999999995,"5min":8.0969999999999995,"15min":8.0969999999999995},"max":{"1min":66274.171000000002,"5min":66779.236000000004,"15min":66779.236000000004},"last":8.4149999999999991}]},{"osd":5,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":27.411999999999999,"5min":22.401,"15min":29.731000000000002},"min":{"1min":11.714,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":281.464,"5min":281.464,"15min":695.39700000000005},"last":17.483000000000001},{"interface":"front","average":{"1min":23.632999999999999,"5min":18.774000000000001,"15min":26.581},"min":{"1min":9.2300000000000004,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":222.58799999999999,"5min":222.58799999999999,"15min":577.46299999999997},"last":10.548}]},{"osd":7,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":25.933,"5min":22.887,"15min":22.504999999999999},"min":{"1min":10.718999999999999,"5min":10.291,"15min":10.291},"max":{"1min":246.16200000000001,"5min":246.16200000000001,"15min":246.16200000000001},"last":18.312999999999999},{"interface":"front","average":{"1min":21.619,"5min":20.536000000000001,"15min":20.823},"min":{"1min":8.0589999999999993,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":173.136,"5min":173.136,"15min":173.136},"last":12.818}]}]},{"osd":5,"up_from":14,"seq":60129542606,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64244,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307596,"statfs":{"total":96636764160,"available":96570978304,"internally_reserved":0,"allocated":11317248,"data_stored":6729942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":5.5460000000000003},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":10.417}]},{"osd":1,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.869,"5min":15.02,"15min":851.14999999999998},"min":{"1min":7.2220000000000004,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":14.837999999999999,"5min":216.709,"15min":42105.105000000003},"last":9.4900000000000002},{"interface":"front","average":{"1min":13.176,"5min":19.975999999999999,"15min":856.74199999999996},"min":{"1min":7.9480000000000004,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":17.890000000000001,"5min":217.929,"15min":42107.510000000002},"last":12.323}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":7.492},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":16.245999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.619,"5min":17.259,"15min":15.612},"min":{"1min":5.8739999999999997,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":16.402000000000001,"5min":149.886,"15min":172.952},"last":13.484999999999999},{"interface":"front","average":{"1min":15.183,"5min":24.073,"15min":18.233000000000001},"min":{"1min":10.442,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":25.030000000000001,"5min":231.691,"15min":231.691},"last":14.223000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:41 2023","interfaces":[{"interface":"back","average":{"1min":19513.460999999999,"5min":8370.6499999999996,"15min":2802.931},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":8.5519999999999996},"max":{"1min":66230.460000000006,"5min":66734.365999999995,"15min":66734.365999999995},"last":8.5039999999999996},{"interface":"front","average":{"1min":19500.227999999999,"5min":8373.2870000000003,"15min":2802.7069999999999},"min":{"1min":8.0199999999999996,"5min":7.282,"15min":7.1840000000000002},"max":{"1min":66215.267999999996,"5min":66719.994999999995,"15min":66719.994999999995},"last":15.576000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.661,"5min":21.093,"15min":2609.1860000000001},"min":{"1min":7.8140000000000001,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":24.298999999999999,"5min":247.161,"15min":74994.611999999994},"last":17.087},{"interface":"front","average":{"1min":13.272,"5min":20.702999999999999,"15min":2611.3249999999998},"min":{"1min":8.0670000000000002,"5min":8.0670000000000002,"15min":8.0670000000000002},"max":{"1min":21.920000000000002,"5min":241.91,"15min":74995.471000000005},"last":16.832000000000001}]},{"osd":7,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":15.432,"5min":20.175999999999998,"15min":23.984999999999999},"min":{"1min":11.771000000000001,"5min":9.0150000000000006,"15min":9.0150000000000006},"max":{"1min":23.018000000000001,"5min":112.215,"15min":112.215},"last":18.408000000000001},{"interface":"front","average":{"1min":15.993,"5min":20.603000000000002,"15min":26.225999999999999},"min":{"1min":11.492000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":22.132999999999999,"5min":103.886,"15min":103.886},"last":11.539}]}]},{"osd":4,"up_from":14,"seq":60129542607,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":9.8640000000000008,"5min":21.134,"15min":14.946},"min":{"1min":7.0199999999999996,"5min":4.8760000000000003,"15min":4.5640000000000001},"max":{"1min":16.805,"5min":302.62700000000001,"15min":302.62700000000001},"last":17.922000000000001},{"interface":"front","average":{"1min":13.249000000000001,"5min":24.818000000000001,"15min":18.748000000000001},"min":{"1min":8.6639999999999997,"5min":5.9379999999999997,"15min":5.9379999999999997},"max":{"1min":18.440999999999999,"5min":289.18200000000002,"15min":289.18200000000002},"last":17.375}]},{"osd":1,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":8.0830000000000002,"5min":26.745999999999999,"15min":1846.819},"min":{"1min":5.8609999999999998,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":12.907,"5min":682.15599999999995,"15min":57448.739999999998},"last":6.0510000000000002},{"interface":"front","average":{"1min":12.319000000000001,"5min":33.448,"15min":1853.625},"min":{"1min":8.2409999999999997,"5min":7.4470000000000001,"15min":7.4169999999999998},"max":{"1min":16.358000000000001,"5min":685.01300000000003,"15min":57479.919999999998},"last":7.694}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":12.050000000000001},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":11.497999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.497999999999999,"5min":28.012,"15min":18.911000000000001},"min":{"1min":8.6999999999999993,"5min":6.048,"15min":6.048},"max":{"1min":17.545000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":12.787000000000001},{"interface":"front","average":{"1min":11.528,"5min":26.763000000000002,"15min":18.524000000000001},"min":{"1min":8.1560000000000006,"5min":6.4699999999999998,"15min":6.4050000000000002},"max":{"1min":18.917999999999999,"5min":609.31899999999996,"15min":609.31899999999996},"last":10.324999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":14.292999999999999,"5min":40.664999999999999,"15min":26.305},"min":{"1min":8.8930000000000007,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":35.802,"5min":607.48199999999997,"15min":607.48199999999997},"last":14.112},{"interface":"front","average":{"1min":13.429,"5min":27.794,"15min":21.152999999999999},"min":{"1min":8.0269999999999992,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":37.572000000000003,"5min":287.95100000000002,"15min":592.32100000000003},"last":16.466000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":12.945,"5min":26.32,"15min":2144.3380000000002},"min":{"1min":7.0549999999999997,"5min":7.0549999999999997,"15min":6.9480000000000004},"max":{"1min":16.335999999999999,"5min":688.24300000000005,"15min":75975.607000000004},"last":8.3059999999999992},{"interface":"front","average":{"1min":13.26,"5min":27.960999999999999,"15min":2143.8119999999999},"min":{"1min":7.8680000000000003,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.018000000000001,"5min":610.89800000000002,"15min":75971.296000000002},"last":10.973000000000001}]},{"osd":7,"last update":"Thu Nov 2 16:12:42 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":142.12799999999999,"15min":67.069000000000003},"min":{"1min":12.108000000000001,"5min":10.411,"15min":10.411},"max":{"1min":17.981000000000002,"5min":611.59799999999996,"15min":611.59799999999996},"last":13.545},{"interface":"front","average":{"1min":13.507999999999999,"5min":140.16900000000001,"15min":57.863},"min":{"1min":8.8960000000000008,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":18.225000000000001,"5min":629.07500000000005,"15min":629.07500000000005},"last":8.984}]}]},{"osd":3,"up_from":736,"seq":3161095930307,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103980,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267860,"statfs":{"total":96636764160,"available":96530288640,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":14.074999999999999,"5min":13.201000000000001,"15min":14.895},"min":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":101.90600000000001,"5min":136.09999999999999,"15min":159.828},"last":5.8639999999999999},{"interface":"front","average":{"1min":12.532,"5min":13.233000000000001,"15min":14.949},"min":{"1min":6.0149999999999997,"5min":4.4000000000000004,"15min":4.4000000000000004},"max":{"1min":43.442999999999998,"5min":153.459,"15min":153.459},"last":9.3409999999999993}]},{"osd":1,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":8.8539999999999992,"5min":13.349,"15min":386.524},"min":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":13.83,"5min":82.028000000000006,"15min":20852.026000000002},"last":8.7590000000000003},{"interface":"front","average":{"1min":12.772,"5min":15.637,"15min":382.35199999999998},"min":{"1min":9.5039999999999996,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":27.445,"5min":141.65000000000001,"15min":20733.266},"last":10.476000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":13.039},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":11.032}]},{"osd":4,"last update":"Thu Nov 2 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":13.74,"5min":21.388999999999999,"15min":22.102},"min":{"1min":8.6859999999999999,"5min":7.7439999999999998,"15min":7.0129999999999999},"max":{"1min":31.504000000000001,"5min":194.845,"15min":201.905},"last":11.545},{"interface":"front","average":{"1min":14.554,"5min":21.510000000000002,"15min":22.814},"min":{"1min":8.1989999999999998,"5min":8.1199999999999992,"15min":6.944},"max":{"1min":33.781999999999996,"5min":196.65600000000001,"15min":210.239},"last":9.9160000000000004}]},{"osd":5,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":18.071999999999999,"5min":21.088000000000001,"15min":24.696000000000002},"min":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":86.810000000000002,"5min":177.97999999999999,"15min":290.07100000000003},"last":16.486000000000001},{"interface":"front","average":{"1min":17.045000000000002,"5min":17.774999999999999,"15min":20.707000000000001},"min":{"1min":9.657,"5min":7.6619999999999999,"15min":6.8220000000000001},"max":{"1min":72.721999999999994,"5min":143.268,"15min":268.577},"last":12.227}]},{"osd":6,"last update":"Thu Nov 2 16:12:39 2023","interfaces":[{"interface":"back","average":{"1min":17.827999999999999,"5min":18.077999999999999,"15min":1643.2470000000001},"min":{"1min":11.323,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":36.439999999999998,"5min":94.352000000000004,"15min":76086.231},"last":13.927},{"interface":"front","average":{"1min":15.978999999999999,"5min":20.367000000000001,"15min":1652.1669999999999},"min":{"1min":9.4280000000000008,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":56.807000000000002,"5min":146.048,"15min":76087.114000000001},"last":15.724}]},{"osd":7,"last update":"Thu Nov 2 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":16.010000000000002,"5min":19.359000000000002,"15min":23.548999999999999},"min":{"1min":10.382,"5min":8.8870000000000005,"15min":8.8870000000000005},"max":{"1min":40.081000000000003,"5min":152.32599999999999,"15min":152.32599999999999},"last":17.158999999999999},{"interface":"front","average":{"1min":16.681999999999999,"5min":19.952000000000002,"15min":24.228000000000002},"min":{"1min":9.5419999999999998,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":39.018000000000001,"5min":140.87299999999999,"15min":140.87299999999999},"last":17.936}]}]},{"osd":7,"up_from":875,"seq":3758096384430,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":10.130000000000001,"5min":13.368,"15min":15.576000000000001},"min":{"1min":6.1070000000000002,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":15.327,"5min":84.426000000000002,"15min":84.426000000000002},"last":6.6079999999999997},{"interface":"front","average":{"1min":8.7460000000000004,"5min":13.207000000000001,"15min":18.576000000000001},"min":{"1min":5.8650000000000002,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":14.715999999999999,"5min":97.984999999999999,"15min":97.984999999999999},"last":8.3710000000000004}]},{"osd":1,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":12.630000000000001,"5min":16.503,"15min":22.439},"min":{"1min":7.5309999999999997,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":16.297000000000001,"5min":178.56,"15min":178.56},"last":11.303000000000001},{"interface":"front","average":{"1min":9.2569999999999997,"5min":12.856999999999999,"15min":15.263},"min":{"1min":6.6589999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":11.67,"5min":66.081999999999994,"15min":66.081999999999994},"last":7.3380000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":10.728},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":18.030999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":14.497999999999999,"5min":20.501000000000001,"15min":19.623000000000001},"min":{"1min":8.9700000000000006,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":25.792000000000002,"5min":253.71000000000001,"15min":253.71000000000001},"last":13.08},{"interface":"front","average":{"1min":12.202,"5min":16.843,"15min":18.457999999999998},"min":{"1min":7.9500000000000002,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":17.898,"5min":101.316,"15min":101.316},"last":16.030000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:12:44 2023","interfaces":[{"interface":"back","average":{"1min":15907.763999999999,"5min":9828.0190000000002,"15min":3302.183},"min":{"1min":8.968,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":65773.679000000004,"5min":66277.987999999998,"15min":66277.987999999998},"last":9.9890000000000008},{"interface":"front","average":{"1min":15924.536,"5min":9833.3089999999993,"15min":3301.9609999999998},"min":{"1min":11.398999999999999,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":65797.989000000001,"5min":66301.380000000005,"15min":66301.380000000005},"last":13.717000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":13.279,"5min":21.350999999999999,"15min":22.777999999999999},"min":{"1min":8.9740000000000002,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":17.827999999999999,"5min":263.65699999999998,"15min":263.65699999999998},"last":17.004999999999999},{"interface":"front","average":{"1min":14.351000000000001,"5min":22.829999999999998,"15min":24.738},"min":{"1min":9.6669999999999998,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":19.327999999999999,"5min":183.96899999999999,"15min":183.96899999999999},"last":11.917}]},{"osd":6,"last update":"Thu Nov 2 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":18.141999999999999,"5min":24.789000000000001,"15min":26.550999999999998},"min":{"1min":10.255000000000001,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":72.542000000000002,"5min":285.56999999999999,"15min":285.56999999999999},"last":12.526999999999999},{"interface":"front","average":{"1min":17.75,"5min":22.82,"15min":23.931999999999999},"min":{"1min":9.7289999999999992,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":28.934000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":15.372999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:13:34.548 INFO:tasks.ceph:pgid 10.16 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.548 INFO:tasks.ceph:pgid 10.17 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.548 INFO:tasks.ceph:pgid 10.14 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.549 INFO:tasks.ceph:pgid 10.15 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.549 INFO:tasks.ceph:pgid 10.12 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.549 INFO:tasks.ceph:pgid 10.13 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.550 INFO:tasks.ceph:pgid 10.10 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.550 INFO:tasks.ceph:pgid 10.11 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.550 INFO:tasks.ceph:pgid 10.1e last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.551 INFO:tasks.ceph:pgid 10.1f last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.551 INFO:tasks.ceph:pgid 10.1c last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.551 INFO:tasks.ceph:pgid 10.1d last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.551 INFO:tasks.ceph:pgid 10.f last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.552 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.552 INFO:tasks.ceph:pgid 10.e last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.552 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.553 INFO:tasks.ceph:pgid 10.d last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.553 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.553 INFO:tasks.ceph:pgid 10.c last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.553 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.554 INFO:tasks.ceph:pgid 10.a last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.554 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.554 INFO:tasks.ceph:pgid 10.9 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.555 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.555 INFO:tasks.ceph:pgid 10.8 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.556 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.557 INFO:tasks.ceph:pgid 10.b last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.557 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-02T15:34:00.052706+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=34, tm_sec=0, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.558 INFO:tasks.ceph:pgid 10.0 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.558 INFO:tasks.ceph:pgid 10.3 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.558 INFO:tasks.ceph:pgid 10.2 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.559 INFO:tasks.ceph:pgid 10.5 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.559 INFO:tasks.ceph:pgid 10.4 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.559 INFO:tasks.ceph:pgid 10.7 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.560 INFO:tasks.ceph:pgid 10.6 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.560 INFO:tasks.ceph:pgid 10.19 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.560 INFO:tasks.ceph:pgid 10.18 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.560 INFO:tasks.ceph:pgid 10.1b last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.561 INFO:tasks.ceph:pgid 10.1a last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:34.561 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-02T16:13:44.474 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:56.628 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.475 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:56.629 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.495 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:40:56.649 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.530 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:56.626 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.531 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:56.627 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.538 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:56.692 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.542 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:56.696 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.552 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:40:56.648 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:44.562 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:40:56.716 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.500 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:01.653 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.502 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:01.655 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.512 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:01.665 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.547 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:01.643 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.548 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:01.644 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.553 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:01.706 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.555 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:01.708 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.558 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:01.654 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:49.568 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:01.721 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:54.562 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:13:54.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.663+0000 7fa2c58ca640 10 monclient: get_monmap_and_config 2023-11-02T16:13:54.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.663+0000 7fa2c58ca640 10 monclient: build_initial_monmap 2023-11-02T16:13:54.665 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.663+0000 7fa2c58ca640 10 monclient: monmap: 2023-11-02T16:13:54.666 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:54.666 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:54.666 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:13:54.664442+0000 2023-11-02T16:13:54.666 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:13:54.664442+0000 2023-11-02T16:13:54.666 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:54.667 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.667 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:13:54.667 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:13:54.667 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:13:54.667 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:13:54.668 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:13:54.668 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 Processor -- start 2023-11-02T16:13:54.670 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 -- start start 2023-11-02T16:13:54.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient: init 2023-11-02T16:13:54.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:54.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2023-11-02T16:13:54.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 0x7fa2c0141800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:54.671 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-b con 0x7fa2c00b4170 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:54.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa2c00a8df0 addr v1:172.21.15.174:6789/0 2023-11-02T16:13:54.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa2c00b13e0 addr v1:172.21.15.175:6789/0 2023-11-02T16:13:54.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00aa3d0 con 0x7fa2c00a8df0 2023-11-02T16:13:54.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00a92d0 con 0x7fa2c00b13e0 2023-11-02T16:13:54.672 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:54.673 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c013f420 con 0x7fa2c00b4170 2023-11-02T16:13:54.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:54.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.664+0000 7fa2c58ca640 10 monclient(hunting): authenticate will time out at 3762.006567s 2023-11-02T16:13:54.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.665+0000 7fa2b7fff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fa2c00a8df0 0x7fa2c0141d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:36182/0 (socket says 172.21.15.174:36182) 2023-11-02T16:13:54.674 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.665+0000 7fa2b7fff640 1 -- 172.21.15.174:0/2679713563 learned_addr learned my addr 172.21.15.174:0/2679713563 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:13:54.675 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.665+0000 7fa2beffd640 1 --2- 172.21.15.174:0/2679713563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 0x7fa2c0141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:54.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): get_auth_request con 0x7fa2c00b4170 auth_method 0 2023-11-02T16:13:54.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:54.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:54.676 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:54.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:54.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:54.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.666+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:54.677 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 <== mon.0 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314114679 0 0) 0x7fa2c00aa3d0 con 0x7fa2c00a8df0 2023-11-02T16:13:54.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 10 monclient(hunting): _negotiate 2023-11-02T16:13:54.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:54.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:54.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 10 monclient(hunting): my global_id is 15846 2023-11-02T16:13:54.678 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.667+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003520 con 0x7fa2c00a8df0 2023-11-02T16:13:54.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 10 monclient(hunting): handle_auth_done global_id 16967 payload 274 2023-11-02T16:13:54.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 1 -- 172.21.15.174:0/2679713563 >> v1:172.21.15.175:6789/0 conn(0x7fa2c00b13e0 legacy=0x7fa2c013e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:54.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 1 -- 172.21.15.174:0/2679713563 >> v1:172.21.15.174:6789/0 conn(0x7fa2c00a8df0 legacy=0x7fa2c0141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:54.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568625385 0 0) 0x7fa2c00a92d0 con 0x7fa2c00b13e0 2023-11-02T16:13:54.679 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 10 monclient: _finish_hunting 0 2023-11-02T16:13:54.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 1 monclient: found mon.noname-b 2023-11-02T16:13:54.680 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:54.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:13:54.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2beffd640 1 -- 172.21.15.174:0/2679713563 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0142450 con 0x7fa2c00b4170 2023-11-02T16:13:54.681 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.668+0000 7fa2be7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-11-02T16:13:54.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.670+0000 7fa2beffd640 1 --2- 172.21.15.174:0/2679713563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 0x7fa2c0141800 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009800 tx=0x7fa2b002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed54d9ff061a11e server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:54.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0031800 con 0x7fa2c00b4170 2023-11-02T16:13:54.688 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:54.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:13:54.689 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: dump: 2023-11-02T16:13:54.689 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:54.689 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:54.689 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.690 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.690 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:54.690 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.691 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:54.691 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:54.691 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:54.692 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: _finish_auth 0 2023-11-02T16:13:54.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: _check_auth_tickets 2023-11-02T16:13:54.692 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:54.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0031960 con 0x7fa2c00b4170 2023-11-02T16:13:54.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:54.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 1 -- 172.21.15.174:0/2679713563 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0031cf0 con 0x7fa2c00b4170 2023-11-02T16:13:54.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2c58ca640 5 monclient: authenticate success, global_id 16967 2023-11-02T16:13:54.693 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2c58ca640 10 monclient: get_monmap_and_config success 2023-11-02T16:13:54.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2c58ca640 4 set_mon_vals no callback set 2023-11-02T16:13:54.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:54.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:13:54.694 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.673+0000 7fa2be7fc640 10 monclient: dump: 2023-11-02T16:13:54.695 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:54.695 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:54.695 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.695 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.695 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:54.697 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.697 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:54.697 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:54.697 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:54.698 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 10 monclient: shutdown 2023-11-02T16:13:54.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:54.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 -- 172.21.15.174:0/2679713563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 msgr2=0x7fa2c0141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:54.698 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/2679713563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 0x7fa2c0141800 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009800 tx=0x7fa2b002b8a0 comp rx=0 tx=0).stop 2023-11-02T16:13:54.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 -- 172.21.15.174:0/2679713563 shutdown_connections 2023-11-02T16:13:54.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/2679713563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b4170 0x7fa2c0141800 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:54.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 -- 172.21.15.174:0/2679713563 >> 172.21.15.174:0/2679713563 conn(0x7fa2c0065d80 msgr2=0x7fa2c0066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:54.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 -- 172.21.15.174:0/2679713563 shutdown_connections 2023-11-02T16:13:54.699 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.674+0000 7fa2c58ca640 1 -- 172.21.15.174:0/2679713563 wait complete. 2023-11-02T16:13:54.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 10 monclient: build_initial_monmap 2023-11-02T16:13:54.700 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 10 monclient: monmap: 2023-11-02T16:13:54.700 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:13:54.700 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:13:54.700 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:13:54.701 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:13:54.701 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:13:54.701 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.702 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:13:54.702 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:13:54.702 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:13:54.702 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 librados: starting msgr at 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 librados: starting objecter 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 Processor -- start 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 -- start start 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 librados: setting wanted keys 2023-11-02T16:13:54.703 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 1 librados: calling monclient init 2023-11-02T16:13:54.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.675+0000 7fa2c58ca640 10 monclient: init 2023-11-02T16:13:54.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient: _reopen_session rank -1 2023-11-02T16:13:54.704 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:13:54.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:54.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-b con 0x7fa2c00a8df0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:13:54.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b13e0 0x7fa2c01da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:54.705 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-c con 0x7fa2c00b13e0 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:13:54.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:54.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): picked mon.noname-a con 0x7fa2c00b4170 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:13:54.706 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:54.707 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01d99e0 con 0x7fa2c00b4170 2023-11-02T16:13:54.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:54.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00ac6e0 con 0x7fa2c00a8df0 2023-11-02T16:13:54.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): start opening mon connection 2023-11-02T16:13:54.708 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01d9bc0 con 0x7fa2c00b13e0 2023-11-02T16:13:54.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): _renew_subs 2023-11-02T16:13:54.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2c58ca640 10 monclient(hunting): authenticate will time out at 3762.017767s 2023-11-02T16:13:54.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2b7fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b13e0 0x7fa2c01da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:54.709 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:54.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:54.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:59046/0 (socket says 172.21.15.174:59046) 2023-11-02T16:13:54.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56960/0 (socket says 172.21.15.174:56960) 2023-11-02T16:13:54.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 1 -- 172.21.15.174:0/3446630189 learned_addr learned my addr 172.21.15.174:0/3446630189 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:13:54.710 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 10 monclient(hunting): get_auth_request con 0x7fa2c00a8df0 auth_method 0 2023-11-02T16:13:54.711 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:54.711 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:54.711 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2beffd640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:54.711 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 10 monclient(hunting): get_auth_request con 0x7fa2c00b4170 auth_method 0 2023-11-02T16:13:54.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:54.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:54.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.676+0000 7fa2bf7fe640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:54.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa2c00b13e0 auth_method 0 2023-11-02T16:13:54.712 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:13:54.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:13:54.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:13:54.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:54.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:54.713 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:54.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:54.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:54.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:54.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:13:54.714 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:13:54.715 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.677+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:13:54.715 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 10 monclient(hunting): handle_auth_done global_id 15849 payload 824 2023-11-02T16:13:54.715 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 -- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b13e0 msgr2=0x7fa2c01da0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:54.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b13e0 0x7fa2c01da0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:54.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 -- 172.21.15.174:0/3446630189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 msgr2=0x7fa2c00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:54.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:54.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 10 monclient: _finish_hunting 0 2023-11-02T16:13:54.716 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 monclient: found mon.noname-a 2023-11-02T16:13:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:13:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.174:3300/0 2023-11-02T16:13:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2bf7fe640 1 -- 172.21.15.174:0/3446630189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0009400 con 0x7fa2c00b4170 2023-11-02T16:13:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.680+0000 7fa2beffd640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T16:13:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.682+0000 7fa2bf7fe640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00b1850 tx=0x7fa2b800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:54.718 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.682+0000 7fa2bcff9640 10 client.?.objecter ms_handle_connect 0x7fa2c00b4170 2023-11-02T16:13:54.718 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.682+0000 7fa2bcff9640 10 client.?.objecter resend_mon_ops 2023-11-02T16:13:54.718 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b70 con 0x7fa2c00b4170 2023-11-02T16:13:54.718 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:54.719 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-11-02T16:13:54.719 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: dump: 2023-11-02T16:13:54.719 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:54.719 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:54.719 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.720 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.720 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:54.720 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.720 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:54.720 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:54.721 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:54.721 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: _finish_auth 0 2023-11-02T16:13:54.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: _check_auth_tickets 2023-11-02T16:13:54.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:13:54.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003cf0 con 0x7fa2c00b4170 2023-11-02T16:13:54.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:13:54.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80052b0 con 0x7fa2c00b4170 2023-11-02T16:13:54.722 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 5 monclient: authenticate success, global_id 15849 2023-11-02T16:13:54.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 10 monclient: _renew_subs 2023-11-02T16:13:54.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:54.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00ae600 con 0x7fa2c00b4170 2023-11-02T16:13:54.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:13:54.723 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-11-02T16:13:54.724 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bcff9640 10 monclient: dump: 2023-11-02T16:13:54.724 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:13:54.724 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:13:54.724 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.724 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:13:54.725 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:13:54.725 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:13:54.725 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:13:54.725 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:13:54.726 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:13:54.726 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:13:54.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2bdffb640 4 set_mon_vals no callback set 2023-11-02T16:13:54.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 10 client.15849.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:13:54.727 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 10 monclient: _renew_subs 2023-11-02T16:13:54.727 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:54.727 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c01da890 con 0x7fa2c00b4170 2023-11-02T16:13:54.727 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.686+0000 7fa2c58ca640 1 librados: init done 2023-11-02T16:13:54.729 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.687+0000 7fa2c58ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:54.729 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.687+0000 7fa2c58ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:54.729 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.687+0000 7fa2c58ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.174:3300/0 2023-11-02T16:13:54.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.687+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c004e040 con 0x7fa2c00b4170 2023-11-02T16:13:54.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.706+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2b8005410 con 0x7fa2c00b4170 2023-11-02T16:13:54.730 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.706+0000 7fa2bcff9640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 0x7fa2a006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:13:54.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.706+0000 7fa2beffd640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 0x7fa2a006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:13:54.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.706+0000 7fa2beffd640 10 monclient: get_auth_request con 0x7fa2a006be10 auth_method 0 2023-11-02T16:13:54.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.707+0000 7fa2beffd640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 0x7fa2a006e2d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000c00 tx=0x7fa2b003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:13:54.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.707+0000 7fa2bcff9640 10 client.15849.objecter ms_handle_connect 0x7fa2a006be10 2023-11-02T16:13:54.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.708+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fa2b8058340 con 0x7fa2c00b4170 2023-11-02T16:13:54.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.708+0000 7fa2bcff9640 10 client.15849.objecter ms_dispatch 0x7fa2c0000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:13:54.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.708+0000 7fa2bcff9640 3 client.15849.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:13:54.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.708+0000 7fa2bcff9640 3 client.15849.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:13:54.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.708+0000 7fa2bcff9640 20 client.15849.objecter dump_active .. 0 homeless 2023-11-02T16:13:54.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.946+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2b8054680 con 0x7fa2c00b4170 2023-11-02T16:13:54.948 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.946+0000 7fa2bcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:13:54.949 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:54.946+0000 7fa2bcff9640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:13:55.126 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.124+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2c01d9d80 con 0x7fa2a006be10 2023-11-02T16:13:55.131 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.129+0000 7fa2bcff9640 1 -- 172.21.15.174:0/3446630189 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152082 (secure 0 0 0) 0x7fa2c01d9d80 con 0x7fa2a006be10 2023-11-02T16:13:55.133 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:13:55.134 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:13:55.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 10 librados: watch_flush enter 2023-11-02T16:13:55.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 10 librados: watch_flush exit 2023-11-02T16:13:55.136 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 20 client.15849.objecter shutdown clearing up homeless session... 2023-11-02T16:13:55.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 10 client.15849.objecter successfully canceled tick 2023-11-02T16:13:55.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 msgr2=0x7fa2a006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:55.137 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 0x7fa2a006e2d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000c00 tx=0x7fa2b003c040 comp rx=0 tx=0).stop 2023-11-02T16:13:55.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 10 monclient: shutdown 2023-11-02T16:13:55.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:13:55.138 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 msgr2=0x7fa2c01d6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:13:55.139 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00b1850 tx=0x7fa2b800cb30 comp rx=0 tx=0).stop 2023-11-02T16:13:55.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 shutdown_connections 2023-11-02T16:13:55.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00b4170 0x7fa2c01d6420 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:55.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fa2a006be10 0x7fa2a006e2d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:55.142 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00b13e0 0x7fa2c01da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:55.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 --2- 172.21.15.174:0/3446630189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2c00a8df0 0x7fa2c00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:13:55.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 >> 172.21.15.174:0/3446630189 conn(0x7fa2c0065d80 msgr2=0x7fa2c00b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:13:55.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 shutdown_connections 2023-11-02T16:13:55.143 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 -- 172.21.15.174:0/3446630189 wait complete. 2023-11-02T16:13:55.144 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:13:55.131+0000 7fa2c58ca640 1 librados: shutdown 2023-11-02T16:13:55.145 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2234,"stamp":"2023-11-02T16:13:53.412169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609908,"kb_used_data":86068,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364812,"statfs":{"total":773094113280,"available":772469567488,"internally_reserved":0,"allocated":88133632,"data_stored":51451729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003741"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":1014,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:42.050272+0000","last_change":"2023-11-02T16:13:42.051930+0000","last_active":"2023-11-02T16:13:42.050272+0000","last_peered":"2023-11-02T16:13:42.050272+0000","last_clean":"2023-11-02T16:13:42.050272+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:42.050272+0000","last_undegraded":"2023-11-02T16:13:42.050272+0000","last_fullsized":"2023-11-02T16:13:42.050272+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:42.049538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:42.049538+0000","last_clean_scrub_stamp":"2023-11-02T16:13:42.049538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:21:40.305486+0000","scrub_duration":0.039569602000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1013,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.544260+0000","last_change":"2023-11-02T16:07:27.307241+0000","last_active":"2023-11-02T16:13:24.544260+0000","last_peered":"2023-11-02T16:13:24.544260+0000","last_clean":"2023-11-02T16:13:24.544260+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:24.544260+0000","last_undegraded":"2023-11-02T16:13:24.544260+0000","last_fullsized":"2023-11-02T16:13:24.544260+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":947,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:47.229021+0000","last_change":"2023-11-02T16:13:47.230619+0000","last_active":"2023-11-02T16:13:47.229021+0000","last_peered":"2023-11-02T16:13:47.229021+0000","last_clean":"2023-11-02T16:13:47.229021+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:47.229021+0000","last_undegraded":"2023-11-02T16:13:47.229021+0000","last_fullsized":"2023-11-02T16:13:47.229021+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:47.228308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:47.228308+0000","last_clean_scrub_stamp":"2023-11-02T16:13:47.228308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:01:59.650918+0000","scrub_duration":0.040728582999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":976,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.609135+0000","last_change":"2023-11-02T15:35:17.345372+0000","last_active":"2023-11-02T16:13:24.609135+0000","last_peered":"2023-11-02T16:13:24.609135+0000","last_clean":"2023-11-02T16:13:24.609135+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:24.609135+0000","last_undegraded":"2023-11-02T16:13:24.609135+0000","last_fullsized":"2023-11-02T16:13:24.609135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":986,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.175163+0000","last_change":"2023-11-02T16:13:41.176774+0000","last_active":"2023-11-02T16:13:41.175163+0000","last_peered":"2023-11-02T16:13:41.175163+0000","last_clean":"2023-11-02T16:13:41.175163+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:41.175163+0000","last_undegraded":"2023-11-02T16:13:41.175163+0000","last_fullsized":"2023-11-02T16:13:41.175163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.174422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.174422+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.174422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:14:12.557347+0000","scrub_duration":0.046185103999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":934,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:37.163987+0000","last_change":"2023-11-02T16:13:37.168167+0000","last_active":"2023-11-02T16:13:37.163987+0000","last_peered":"2023-11-02T16:13:37.163987+0000","last_clean":"2023-11-02T16:13:37.163987+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:37.163987+0000","last_undegraded":"2023-11-02T16:13:37.163987+0000","last_fullsized":"2023-11-02T16:13:37.163987+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:37.163062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:37.163062+0000","last_clean_scrub_stamp":"2023-11-02T16:13:37.163062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:04:26.268836+0000","scrub_duration":0.065834622999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1012,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.116761+0000","last_change":"2023-11-02T16:13:44.118515+0000","last_active":"2023-11-02T16:13:44.116761+0000","last_peered":"2023-11-02T16:13:44.116761+0000","last_clean":"2023-11-02T16:13:44.116761+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:44.116761+0000","last_undegraded":"2023-11-02T16:13:44.116761+0000","last_fullsized":"2023-11-02T16:13:44.116761+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.116015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.116015+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.116015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:46:01.480651+0000","scrub_duration":0.040346910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":986,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.462604+0000","last_change":"2023-11-02T16:13:32.468370+0000","last_active":"2023-11-02T16:13:32.462604+0000","last_peered":"2023-11-02T16:13:32.462604+0000","last_clean":"2023-11-02T16:13:32.462604+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:32.462604+0000","last_undegraded":"2023-11-02T16:13:32.462604+0000","last_fullsized":"2023-11-02T16:13:32.462604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.461412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.461412+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.461412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:17:47.255889+0000","scrub_duration":0.072050043999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":975,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:38.012415+0000","last_change":"2023-11-02T16:13:38.014259+0000","last_active":"2023-11-02T16:13:38.012415+0000","last_peered":"2023-11-02T16:13:38.012415+0000","last_clean":"2023-11-02T16:13:38.012415+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:38.012415+0000","last_undegraded":"2023-11-02T16:13:38.012415+0000","last_fullsized":"2023-11-02T16:13:38.012415+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:38.011629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:38.011629+0000","last_clean_scrub_stamp":"2023-11-02T16:13:38.011629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:37:20.779523+0000","scrub_duration":0.071776143000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:42.254430+0000","last_change":"2023-11-02T16:13:42.256054+0000","last_active":"2023-11-02T16:13:42.254430+0000","last_peered":"2023-11-02T16:13:42.254430+0000","last_clean":"2023-11-02T16:13:42.254430+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:42.254430+0000","last_undegraded":"2023-11-02T16:13:42.254430+0000","last_fullsized":"2023-11-02T16:13:42.254430+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:42.253698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:42.253698+0000","last_clean_scrub_stamp":"2023-11-02T16:13:42.253698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:07:54.847830+0000","scrub_duration":0.050904000999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1026,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.240445+0000","last_change":"2023-11-02T16:13:41.240445+0000","last_active":"2023-11-02T16:13:41.240445+0000","last_peered":"2023-11-02T16:13:41.240445+0000","last_clean":"2023-11-02T16:13:41.240445+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:41.240445+0000","last_undegraded":"2023-11-02T16:13:41.240445+0000","last_fullsized":"2023-11-02T16:13:41.240445+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.234782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.234782+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.234782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:00:16.955119+0000","scrub_duration":0.037739861,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1013,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:24.537110+0000","last_change":"2023-11-02T16:13:17.189587+0000","last_active":"2023-11-02T16:13:24.537110+0000","last_peered":"2023-11-02T16:13:24.537110+0000","last_clean":"2023-11-02T16:13:24.537110+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:24.537110+0000","last_undegraded":"2023-11-02T16:13:24.537110+0000","last_fullsized":"2023-11-02T16:13:24.537110+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T15:35:07.265583+0000","last_clean_scrub_stamp":"2023-11-02T15:35:07.265583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":999,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:45.912186+0000","last_change":"2023-11-02T16:13:45.913860+0000","last_active":"2023-11-02T16:13:45.912186+0000","last_peered":"2023-11-02T16:13:45.912186+0000","last_clean":"2023-11-02T16:13:45.912186+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:45.912186+0000","last_undegraded":"2023-11-02T16:13:45.912186+0000","last_fullsized":"2023-11-02T16:13:45.912186+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:45.911467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:45.911467+0000","last_clean_scrub_stamp":"2023-11-02T16:13:45.911467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:26:44.521107+0000","scrub_duration":0.066455971000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:43.947682+0000","last_change":"2023-11-02T16:13:43.952967+0000","last_active":"2023-11-02T16:13:43.947682+0000","last_peered":"2023-11-02T16:13:43.947682+0000","last_clean":"2023-11-02T16:13:43.947682+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:43.947682+0000","last_undegraded":"2023-11-02T16:13:43.947682+0000","last_fullsized":"2023-11-02T16:13:43.947682+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:43.946966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:43.946966+0000","last_clean_scrub_stamp":"2023-11-02T16:13:43.946966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:34:12.457157+0000","scrub_duration":0.033136158999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":990,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.062291+0000","last_change":"2023-11-02T16:13:41.067307+0000","last_active":"2023-11-02T16:13:41.062291+0000","last_peered":"2023-11-02T16:13:41.062291+0000","last_clean":"2023-11-02T16:13:41.062291+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:41.062291+0000","last_undegraded":"2023-11-02T16:13:41.062291+0000","last_fullsized":"2023-11-02T16:13:41.062291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.061561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.061561+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.061561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:09:22.648006+0000","scrub_duration":0.047085142000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":994,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.930206+0000","last_change":"2023-11-02T16:13:32.930206+0000","last_active":"2023-11-02T16:13:32.930206+0000","last_peered":"2023-11-02T16:13:32.930206+0000","last_clean":"2023-11-02T16:13:32.930206+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:32.930206+0000","last_undegraded":"2023-11-02T16:13:32.930206+0000","last_fullsized":"2023-11-02T16:13:32.930206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.927710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.927710+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.927710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:05:46.839406+0000","scrub_duration":0.045056061000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1004,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:43.111816+0000","last_change":"2023-11-02T16:13:43.113486+0000","last_active":"2023-11-02T16:13:43.111816+0000","last_peered":"2023-11-02T16:13:43.111816+0000","last_clean":"2023-11-02T16:13:43.111816+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:43.111816+0000","last_undegraded":"2023-11-02T16:13:43.111816+0000","last_fullsized":"2023-11-02T16:13:43.111816+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:43.111090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:43.111090+0000","last_clean_scrub_stamp":"2023-11-02T16:13:43.111090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:51:40.525837+0000","scrub_duration":0.057244171000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1005,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.934723+0000","last_change":"2023-11-02T16:13:41.936357+0000","last_active":"2023-11-02T16:13:41.934723+0000","last_peered":"2023-11-02T16:13:41.934723+0000","last_clean":"2023-11-02T16:13:41.934723+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:41.934723+0000","last_undegraded":"2023-11-02T16:13:41.934723+0000","last_fullsized":"2023-11-02T16:13:41.934723+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.933984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.933984+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.933984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:02:39.382548+0000","scrub_duration":0.028975570999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1028,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:38.283506+0000","last_change":"2023-11-02T16:13:38.285096+0000","last_active":"2023-11-02T16:13:38.283506+0000","last_peered":"2023-11-02T16:13:38.283506+0000","last_clean":"2023-11-02T16:13:38.283506+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:38.283506+0000","last_undegraded":"2023-11-02T16:13:38.283506+0000","last_fullsized":"2023-11-02T16:13:38.283506+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:38.282778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:38.282778+0000","last_clean_scrub_stamp":"2023-11-02T16:13:38.282778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:47:12.523508+0000","scrub_duration":0.027973423000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":967,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.986922+0000","last_change":"2023-11-02T16:13:32.986922+0000","last_active":"2023-11-02T16:13:32.986922+0000","last_peered":"2023-11-02T16:13:32.986922+0000","last_clean":"2023-11-02T16:13:32.986922+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:32.986922+0000","last_undegraded":"2023-11-02T16:13:32.986922+0000","last_fullsized":"2023-11-02T16:13:32.986922+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.984352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.984352+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.984352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:23:32.900645+0000","scrub_duration":0.034194417999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":546,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:40.140806+0000","last_change":"2023-11-02T16:13:40.142587+0000","last_active":"2023-11-02T16:13:40.140806+0000","last_peered":"2023-11-02T16:13:40.140806+0000","last_clean":"2023-11-02T16:13:40.140806+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:40.140806+0000","last_undegraded":"2023-11-02T16:13:40.140806+0000","last_fullsized":"2023-11-02T16:13:40.140806+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:40.140073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:40.140073+0000","last_clean_scrub_stamp":"2023-11-02T16:13:40.140073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:34:40.870971+0000","scrub_duration":0.046347369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1020,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:31.074304+0000","last_change":"2023-11-02T16:13:31.076061+0000","last_active":"2023-11-02T16:13:31.074304+0000","last_peered":"2023-11-02T16:13:31.074304+0000","last_clean":"2023-11-02T16:13:31.074304+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:31.074304+0000","last_undegraded":"2023-11-02T16:13:31.074304+0000","last_fullsized":"2023-11-02T16:13:31.074304+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:31.073530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:31.073530+0000","last_clean_scrub_stamp":"2023-11-02T16:13:31.073530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:28:08.702885+0000","scrub_duration":0.03585907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":991,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.153704+0000","last_change":"2023-11-02T16:13:34.158824+0000","last_active":"2023-11-02T16:13:34.153704+0000","last_peered":"2023-11-02T16:13:34.153704+0000","last_clean":"2023-11-02T16:13:34.153704+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:34.153704+0000","last_undegraded":"2023-11-02T16:13:34.153704+0000","last_fullsized":"2023-11-02T16:13:34.153704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.152960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.152960+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.152960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:04:12.998703+0000","scrub_duration":0.086841235000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:39.176671+0000","last_change":"2023-11-02T16:13:39.176671+0000","last_active":"2023-11-02T16:13:39.176671+0000","last_peered":"2023-11-02T16:13:39.176671+0000","last_clean":"2023-11-02T16:13:39.176671+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:39.176671+0000","last_undegraded":"2023-11-02T16:13:39.176671+0000","last_fullsized":"2023-11-02T16:13:39.176671+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:39.172970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:39.172970+0000","last_clean_scrub_stamp":"2023-11-02T16:13:39.172970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:44.770192+0000","scrub_duration":0.066176944000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":949,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.991167+0000","last_change":"2023-11-02T16:13:44.993038+0000","last_active":"2023-11-02T16:13:44.991167+0000","last_peered":"2023-11-02T16:13:44.991167+0000","last_clean":"2023-11-02T16:13:44.991167+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:44.991167+0000","last_undegraded":"2023-11-02T16:13:44.991167+0000","last_fullsized":"2023-11-02T16:13:44.991167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.990415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.990415+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.990415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:49:40.351558+0000","scrub_duration":0.036013218999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1006,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:30.029645+0000","last_change":"2023-11-02T16:13:30.031726+0000","last_active":"2023-11-02T16:13:30.029645+0000","last_peered":"2023-11-02T16:13:30.029645+0000","last_clean":"2023-11-02T16:13:30.029645+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:30.029645+0000","last_undegraded":"2023-11-02T16:13:30.029645+0000","last_fullsized":"2023-11-02T16:13:30.029645+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:30.028859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:30.028859+0000","last_clean_scrub_stamp":"2023-11-02T16:13:30.028859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:23:54.699449+0000","scrub_duration":0.03822644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":987,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:49.235869+0000","last_change":"2023-11-02T16:13:49.237577+0000","last_active":"2023-11-02T16:13:49.235869+0000","last_peered":"2023-11-02T16:13:49.235869+0000","last_clean":"2023-11-02T16:13:49.235869+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:49.235869+0000","last_undegraded":"2023-11-02T16:13:49.235869+0000","last_fullsized":"2023-11-02T16:13:49.235869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:49.235124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:49.235124+0000","last_clean_scrub_stamp":"2023-11-02T16:13:49.235124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:05:36.160979+0000","scrub_duration":0.059997438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1031,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.061347+0000","last_change":"2023-11-02T16:13:34.063784+0000","last_active":"2023-11-02T16:13:34.061347+0000","last_peered":"2023-11-02T16:13:34.061347+0000","last_clean":"2023-11-02T16:13:34.061347+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:34.061347+0000","last_undegraded":"2023-11-02T16:13:34.061347+0000","last_fullsized":"2023-11-02T16:13:34.061347+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.060576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.060576+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.060576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:43:43.380598+0000","scrub_duration":0.031708978999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1030'78","reported_seq":1101,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:30.530233+0000","last_change":"2023-11-02T16:13:30.536191+0000","last_active":"2023-11-02T16:13:30.530233+0000","last_peered":"2023-11-02T16:13:30.530233+0000","last_clean":"2023-11-02T16:13:30.530233+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:30.530233+0000","last_undegraded":"2023-11-02T16:13:30.530233+0000","last_fullsized":"2023-11-02T16:13:30.530233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'78","last_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_deep_scrub":"1030'78","last_deep_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_clean_scrub_stamp":"2023-11-02T16:13:30.528744+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:03:01.384002+0000","scrub_duration":0.116374897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"1030'8","reported_seq":1032,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:29.185182+0000","last_change":"2023-11-02T16:13:29.187351+0000","last_active":"2023-11-02T16:13:29.185182+0000","last_peered":"2023-11-02T16:13:29.185182+0000","last_clean":"2023-11-02T16:13:29.185182+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:29.185182+0000","last_undegraded":"2023-11-02T16:13:29.185182+0000","last_fullsized":"2023-11-02T16:13:29.185182+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'8","last_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_deep_scrub":"1030'8","last_deep_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_clean_scrub_stamp":"2023-11-02T16:13:29.183095+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:20:07.879623+0000","scrub_duration":0.12634673699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1016,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:36.223251+0000","last_change":"2023-11-02T16:13:36.229119+0000","last_active":"2023-11-02T16:13:36.223251+0000","last_peered":"2023-11-02T16:13:36.223251+0000","last_clean":"2023-11-02T16:13:36.223251+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:36.223251+0000","last_undegraded":"2023-11-02T16:13:36.223251+0000","last_fullsized":"2023-11-02T16:13:36.223251+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:36.221991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:36.221991+0000","last_clean_scrub_stamp":"2023-11-02T16:13:36.221991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:12:00.805971+0000","scrub_duration":0.038230550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1018,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:36.917516+0000","last_change":"2023-11-02T16:13:36.917516+0000","last_active":"2023-11-02T16:13:36.917516+0000","last_peered":"2023-11-02T16:13:36.917516+0000","last_clean":"2023-11-02T16:13:36.917516+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:36.917516+0000","last_undegraded":"2023-11-02T16:13:36.917516+0000","last_fullsized":"2023-11-02T16:13:36.917516+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:36.915123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:36.915123+0000","last_clean_scrub_stamp":"2023-11-02T16:13:36.915123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:04:42.967961+0000","scrub_duration":0.036527178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":976,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.998609+0000","last_change":"2023-11-02T16:13:35.000385+0000","last_active":"2023-11-02T16:13:34.998609+0000","last_peered":"2023-11-02T16:13:34.998609+0000","last_clean":"2023-11-02T16:13:34.998609+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:34.998609+0000","last_undegraded":"2023-11-02T16:13:34.998609+0000","last_fullsized":"2023-11-02T16:13:34.998609+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.997831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.997831+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.997831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:37:45.627987+0000","scrub_duration":0.036573472000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:48.106734+0000","last_change":"2023-11-02T16:13:48.113388+0000","last_active":"2023-11-02T16:13:48.106734+0000","last_peered":"2023-11-02T16:13:48.106734+0000","last_clean":"2023-11-02T16:13:48.106734+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:48.106734+0000","last_undegraded":"2023-11-02T16:13:48.106734+0000","last_fullsized":"2023-11-02T16:13:48.106734+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:48.105686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:48.105686+0000","last_clean_scrub_stamp":"2023-11-02T16:13:48.105686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:08:39.054141+0000","scrub_duration":0.043874059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":992,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.884540+0000","last_change":"2023-11-02T16:13:34.884540+0000","last_active":"2023-11-02T16:13:34.884540+0000","last_peered":"2023-11-02T16:13:34.884540+0000","last_clean":"2023-11-02T16:13:34.884540+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:34.884540+0000","last_undegraded":"2023-11-02T16:13:34.884540+0000","last_fullsized":"2023-11-02T16:13:34.884540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.878451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.878451+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.878451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:17:12.137889+0000","scrub_duration":0.031321511000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1000,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:47.887241+0000","last_change":"2023-11-02T16:13:47.894547+0000","last_active":"2023-11-02T16:13:47.887241+0000","last_peered":"2023-11-02T16:13:47.887241+0000","last_clean":"2023-11-02T16:13:47.887241+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:47.887241+0000","last_undegraded":"2023-11-02T16:13:47.887241+0000","last_fullsized":"2023-11-02T16:13:47.887241+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:47.886446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:47.886446+0000","last_clean_scrub_stamp":"2023-11-02T16:13:47.886446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:56:34.008046+0000","scrub_duration":0.054713576999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1001,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.910862+0000","last_change":"2023-11-02T16:13:44.918964+0000","last_active":"2023-11-02T16:13:44.910862+0000","last_peered":"2023-11-02T16:13:44.910862+0000","last_clean":"2023-11-02T16:13:44.910862+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:44.910862+0000","last_undegraded":"2023-11-02T16:13:44.910862+0000","last_fullsized":"2023-11-02T16:13:44.910862+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.910100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.910100+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.910100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:11:37.367165+0000","scrub_duration":0.037239766000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":947,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:46.185571+0000","last_change":"2023-11-02T16:13:46.187387+0000","last_active":"2023-11-02T16:13:46.185571+0000","last_peered":"2023-11-02T16:13:46.185571+0000","last_clean":"2023-11-02T16:13:46.185571+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:46.185571+0000","last_undegraded":"2023-11-02T16:13:46.185571+0000","last_fullsized":"2023-11-02T16:13:46.185571+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:46.184801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:46.184801+0000","last_clean_scrub_stamp":"2023-11-02T16:13:46.184801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:12:41.535289+0000","scrub_duration":0.046431591000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:46.105603+0000","last_change":"2023-11-02T16:13:46.107294+0000","last_active":"2023-11-02T16:13:46.105603+0000","last_peered":"2023-11-02T16:13:46.105603+0000","last_clean":"2023-11-02T16:13:46.105603+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:46.105603+0000","last_undegraded":"2023-11-02T16:13:46.105603+0000","last_fullsized":"2023-11-02T16:13:46.105603+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:46.104828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:46.104828+0000","last_clean_scrub_stamp":"2023-11-02T16:13:46.104828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:46:36.744296+0000","scrub_duration":0.045205897000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:37.991837+0000","last_change":"2023-11-02T16:13:37.997167+0000","last_active":"2023-11-02T16:13:37.991837+0000","last_peered":"2023-11-02T16:13:37.991837+0000","last_clean":"2023-11-02T16:13:37.991837+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:37.991837+0000","last_undegraded":"2023-11-02T16:13:37.991837+0000","last_fullsized":"2023-11-02T16:13:37.991837+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:37.991050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:37.991050+0000","last_clean_scrub_stamp":"2023-11-02T16:13:37.991050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:03:55.186070+0000","scrub_duration":0.059452926000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1011,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:40.207732+0000","last_change":"2023-11-02T16:13:40.209753+0000","last_active":"2023-11-02T16:13:40.207732+0000","last_peered":"2023-11-02T16:13:40.207732+0000","last_clean":"2023-11-02T16:13:40.207732+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:40.207732+0000","last_undegraded":"2023-11-02T16:13:40.207732+0000","last_fullsized":"2023-11-02T16:13:40.207732+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:40.206988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:40.206988+0000","last_clean_scrub_stamp":"2023-11-02T16:13:40.206988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:15:17.710464+0000","scrub_duration":0.037045162,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544257,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6332119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":8.1720000000000006},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":8.9009999999999998}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":7.2309999999999999},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":11.706}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":13.211},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":10.961}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":12.675000000000001},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":14.016999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":15.252000000000001},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":17.940000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":15.987},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":16.678999999999998}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":19.280000000000001},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":17.300000000000001}]}]},{"osd":1,"up_from":16,"seq":68719477193,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104372,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267468,"statfs":{"total":96636764160,"available":96529887232,"internally_reserved":0,"allocated":11317248,"data_stored":6729954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":8.7509999999999994,"5min":11.885,"15min":17.582000000000001},"min":{"1min":4.5650000000000004,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":13.523999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":6.984},{"interface":"front","average":{"1min":8.1389999999999993,"5min":12.238,"15min":18.489999999999998},"min":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":11.058999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":11.491}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":17.109000000000002},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":8.9209999999999994}]},{"osd":3,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":12.576000000000001,"5min":13.050000000000001,"15min":21.103999999999999},"min":{"1min":6.8769999999999998,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":19.425000000000001,"5min":64.453000000000003,"15min":549.92200000000003},"last":10.677},{"interface":"front","average":{"1min":12.166,"5min":15.949999999999999,"15min":24.456},"min":{"1min":7.3369999999999997,"5min":7.3369999999999997,"15min":7.3369999999999997},"max":{"1min":28.93,"5min":108.244,"15min":542.60500000000002},"last":12.5}]},{"osd":4,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":13.579000000000001,"5min":109.658,"15min":49.033000000000001},"min":{"1min":9.391,"5min":7.8819999999999997,"15min":6.1989999999999998},"max":{"1min":27.463000000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":13.472},{"interface":"front","average":{"1min":13.228999999999999,"5min":110.251,"15min":50.997999999999998},"min":{"1min":8.8130000000000006,"5min":7.173,"15min":5.8680000000000003},"max":{"1min":20.852,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":15.259}]},{"osd":5,"last update":"Thu Nov 2 16:12:53 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":16.436,"15min":25.068999999999999},"min":{"1min":8.0990000000000002,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":16.541,"5min":71.225999999999999,"15min":529.93499999999995},"last":19.651},{"interface":"front","average":{"1min":15.509,"5min":19.974,"15min":26.859000000000002},"min":{"1min":9.5839999999999996,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":23.559999999999999,"5min":122.14,"15min":555.77700000000004},"last":17.722000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":14.141,"5min":20.727,"15min":1552.6310000000001},"min":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1400000000000006},"max":{"1min":21.079000000000001,"5min":245.36500000000001,"15min":73455.679000000004},"last":20.422000000000001},{"interface":"front","average":{"1min":14.130000000000001,"5min":16.395,"15min":1549.3979999999999},"min":{"1min":9.9269999999999996,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":21.965,"5min":124.36499999999999,"15min":73453.190000000002},"last":18.367999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:13:37 2023","interfaces":[{"interface":"back","average":{"1min":13.800000000000001,"5min":16.815000000000001,"15min":20.478999999999999},"min":{"1min":6.8570000000000002,"5min":6.8570000000000002,"15min":6.8570000000000002},"max":{"1min":21.863,"5min":91.019999999999996,"15min":91.019999999999996},"last":16.542000000000002},{"interface":"front","average":{"1min":17.292999999999999,"5min":20.710999999999999,"15min":23.553999999999998},"min":{"1min":8.6479999999999997,"5min":8.6479999999999997,"15min":8.6479999999999997},"max":{"1min":25.082999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":21.289000000000001}]}]},{"osd":0,"up_from":16,"seq":68719477217,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81900,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289940,"statfs":{"total":96636764160,"available":96552898560,"internally_reserved":0,"allocated":10915840,"data_stored":6331258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:12:55 2023","interfaces":[{"interface":"back","average":{"1min":8.7309999999999999,"5min":13.488,"15min":77.375},"min":{"1min":5.3490000000000002,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.488,"5min":107.59999999999999,"15min":5781.2129999999997},"last":13.1},{"interface":"front","average":{"1min":8.4719999999999995,"5min":11.718,"15min":74.570999999999998},"min":{"1min":6.2249999999999996,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":13.672000000000001,"5min":73.094999999999999,"15min":5443.4889999999996},"last":6.8780000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.186},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":8.266}]},{"osd":3,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":16.105,"5min":15.993,"15min":17.158000000000001},"min":{"1min":8.7569999999999997,"5min":7.9219999999999997,"15min":7.46},"max":{"1min":40.524999999999999,"5min":83.388000000000005,"15min":234.29300000000001},"last":10.845000000000001},{"interface":"front","average":{"1min":15.180999999999999,"5min":15.614000000000001,"15min":15.147},"min":{"1min":6.9050000000000002,"5min":6.9050000000000002,"15min":6.6070000000000002},"max":{"1min":62.927,"5min":124.72199999999999,"15min":151.30099999999999},"last":11.474}]},{"osd":4,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":12.595000000000001,"5min":17483.812999999998,"15min":5839.665},"min":{"1min":7.7169999999999996,"5min":7.7169999999999996,"15min":6.3200000000000003},"max":{"1min":21.286000000000001,"5min":66161.077000000005,"15min":66161.077000000005},"last":13.923},{"interface":"front","average":{"1min":16.332000000000001,"5min":17488.550999999999,"15min":5842.2960000000003},"min":{"1min":11.327,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":34.182000000000002,"5min":66163.926000000007,"15min":66163.926000000007},"last":14.526999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":15.74},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":17.824000000000002}]},{"osd":6,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":21.327999999999999,"5min":20.154,"15min":1772.442},"min":{"1min":15.074,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":92.451999999999998,"5min":92.451999999999998,"15min":71055.725999999995},"last":17.076000000000001},{"interface":"front","average":{"1min":14.882,"5min":15.442,"15min":1770.1980000000001},"min":{"1min":9.7959999999999994,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":28.565999999999999,"5min":85.048000000000002,"15min":71055.100000000006},"last":12.359}]},{"osd":7,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":19.626000000000001,"5min":22.398,"15min":29.359999999999999},"min":{"1min":13.561999999999999,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":40.588000000000001,"5min":110.238,"15min":205.88999999999999},"last":18.523},{"interface":"front","average":{"1min":12.699999999999999,"5min":14.935,"15min":22.003},"min":{"1min":8.0090000000000003,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":34.991,"5min":84.650999999999996,"15min":155.96600000000001},"last":15.182}]}]},{"osd":6,"up_from":387,"seq":1662152343960,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":8.3919999999999995,"5min":14.112,"15min":20.099},"min":{"1min":4.508,"5min":4.508,"15min":4.508},"max":{"1min":13.715,"5min":208.60499999999999,"15min":572.92100000000005},"last":14.103},{"interface":"front","average":{"1min":10.018000000000001,"5min":13.913,"15min":21.109000000000002},"min":{"1min":5.3300000000000001,"5min":4.7539999999999996,"15min":4.0419999999999998},"max":{"1min":19.815000000000001,"5min":172.51499999999999,"15min":630.37900000000002},"last":7.5640000000000001}]},{"osd":1,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":12.505000000000001,"5min":16.201000000000001,"15min":1146.796},"min":{"1min":6.8890000000000002,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":56.890000000000001,"5min":143.55699999999999,"15min":41558.328999999998},"last":8.3019999999999996},{"interface":"front","average":{"1min":11.218999999999999,"5min":16.899999999999999,"15min":1147.8579999999999},"min":{"1min":6.0599999999999996,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":20.495999999999999,"5min":176.839,"15min":41560.286999999997},"last":16.068999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":14.821},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":10.929}]},{"osd":3,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.462,"5min":22.088999999999999,"15min":24.111000000000001},"min":{"1min":8.4540000000000006,"5min":8.4540000000000006,"15min":6.4480000000000004},"max":{"1min":55.115000000000002,"5min":260.11700000000002,"15min":633.899},"last":11.656000000000001},{"interface":"front","average":{"1min":12.683,"5min":18.876000000000001,"15min":24.088999999999999},"min":{"1min":8.0109999999999992,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":17.535,"5min":188.101,"15min":582.32399999999996},"last":10.215}]},{"osd":4,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":14.535,"5min":4461.1090000000004,"15min":1505.2059999999999},"min":{"1min":9.8070000000000004,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":21.734000000000002,"5min":67109.214999999997,"15min":67109.214999999997},"last":16.655000000000001},{"interface":"front","average":{"1min":11.736000000000001,"5min":4422.7439999999997,"15min":1494.021},"min":{"1min":6.992,"5min":6.992,"15min":6.992},"max":{"1min":20.512,"5min":66779.236000000004,"15min":66779.236000000004},"last":9.0150000000000006}]},{"osd":5,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.187000000000001,"5min":21.879999999999999,"15min":29.353000000000002},"min":{"1min":7.9299999999999997,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":54.112000000000002,"5min":281.464,"15min":695.39700000000005},"last":12.353},{"interface":"front","average":{"1min":13.478,"5min":18.571000000000002,"15min":26.082000000000001},"min":{"1min":8.2400000000000002,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":22.428999999999998,"5min":222.58799999999999,"15min":577.46299999999997},"last":18.021999999999998}]},{"osd":7,"last update":"Thu Nov 2 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":17.585000000000001,"5min":21.940999999999999,"15min":22.190000000000001},"min":{"1min":11.49,"5min":10.291,"15min":10.291},"max":{"1min":50.445999999999998,"5min":246.16200000000001,"15min":246.16200000000001},"last":12.868},{"interface":"front","average":{"1min":15.438000000000001,"5min":19.43,"15min":20.454000000000001},"min":{"1min":10.507999999999999,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":18.666,"5min":173.136,"15min":173.136},"last":15.427}]}]},{"osd":5,"up_from":14,"seq":60129542610,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64244,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307596,"statfs":{"total":96636764160,"available":96570978304,"internally_reserved":0,"allocated":11317248,"data_stored":6729942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":5.5439999999999996},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":9.6099999999999994}]},{"osd":1,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.869,"5min":15.02,"15min":851.14999999999998},"min":{"1min":7.2220000000000004,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":14.837999999999999,"5min":216.709,"15min":42105.105000000003},"last":6.96},{"interface":"front","average":{"1min":13.176,"5min":19.975999999999999,"15min":856.74199999999996},"min":{"1min":7.9480000000000004,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":17.890000000000001,"5min":217.929,"15min":42107.510000000002},"last":12.226000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":9.9570000000000007},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":12.925000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:12:51 2023","interfaces":[{"interface":"back","average":{"1min":10.619,"5min":17.259,"15min":15.612},"min":{"1min":5.8739999999999997,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":16.402000000000001,"5min":149.886,"15min":172.952},"last":10.911},{"interface":"front","average":{"1min":15.183,"5min":24.073,"15min":18.233000000000001},"min":{"1min":10.442,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":25.030000000000001,"5min":231.691,"15min":231.691},"last":11.519}]},{"osd":4,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.256,"5min":8367.366,"15min":2801.4569999999999},"min":{"1min":8.5039999999999996,"5min":8.5039999999999996,"15min":8.5039999999999996},"max":{"1min":52.700000000000003,"5min":66734.365999999995,"15min":66734.365999999995},"last":14.993},{"interface":"front","average":{"1min":18.460999999999999,"5min":8369.3950000000004,"15min":2801.703},"min":{"1min":6.7409999999999997,"5min":6.7409999999999997,"15min":6.7409999999999997},"max":{"1min":105.985,"5min":66719.994999999995,"15min":66719.994999999995},"last":16.335999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":15.371,"5min":19.039999999999999,"15min":2608.2600000000002},"min":{"1min":8.1110000000000007,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":31.577000000000002,"5min":247.161,"15min":74994.611999999994},"last":15.696},{"interface":"front","average":{"1min":13.83,"5min":18.111000000000001,"15min":2610.2930000000001},"min":{"1min":7.4080000000000004,"5min":7.4080000000000004,"15min":7.4080000000000004},"max":{"1min":21.225000000000001,"5min":241.91,"15min":74995.471000000005},"last":13.6}]},{"osd":7,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":20.102,"5min":19.018000000000001,"15min":23.599},"min":{"1min":10.536,"5min":10.536,"15min":9.0150000000000006},"max":{"1min":106.866,"5min":106.866,"15min":112.215},"last":18.023},{"interface":"front","average":{"1min":15.651999999999999,"5min":17.925999999999998,"15min":25.334},"min":{"1min":10.284000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":28.260000000000002,"5min":93.725999999999999,"15min":103.886},"last":16.968}]}]},{"osd":4,"up_from":14,"seq":60129542611,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":11.961,"5min":21.25,"15min":14.602},"min":{"1min":5.4219999999999997,"5min":5.0030000000000001,"15min":4.5640000000000001},"max":{"1min":74.665000000000006,"5min":302.62700000000001,"15min":302.62700000000001},"last":7.9939999999999998},{"interface":"front","average":{"1min":16.593,"5min":24.837,"15min":18.509},"min":{"1min":8.8499999999999996,"5min":8.125,"15min":5.9379999999999997},"max":{"1min":91.230999999999995,"5min":289.18200000000002,"15min":289.18200000000002},"last":8.8499999999999996}]},{"osd":1,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":8.1400000000000006,"5min":26.207999999999998,"15min":894.40300000000002},"min":{"1min":5.5780000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":10.657999999999999,"5min":682.15599999999995,"15min":38151.266000000003},"last":10.337999999999999},{"interface":"front","average":{"1min":11.291,"5min":32.579999999999998,"15min":900.13999999999999},"min":{"1min":7.1660000000000004,"5min":7.1660000000000004,"15min":7.1660000000000004},"max":{"1min":16.356000000000002,"5min":685.01300000000003,"15min":38172.237000000001},"last":12.178000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":13.428000000000001},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":5.7919999999999998}]},{"osd":3,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":14.381,"5min":27.777999999999999,"15min":18.989999999999998},"min":{"1min":7.4180000000000001,"5min":6.048,"15min":6.048},"max":{"1min":46.685000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":14.019},{"interface":"front","average":{"1min":14.340999999999999,"5min":26.373000000000001,"15min":18.59},"min":{"1min":6.4000000000000004,"5min":6.4000000000000004,"15min":6.4000000000000004},"max":{"1min":65.313000000000002,"5min":609.31899999999996,"15min":609.31899999999996},"last":9.4060000000000006}]},{"osd":5,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":22.913,"5min":41.131,"15min":24.827999999999999},"min":{"1min":8.8740000000000006,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":192.56700000000001,"5min":607.48199999999997,"15min":607.48199999999997},"last":14.622},{"interface":"front","average":{"1min":21.675000000000001,"5min":28.567,"15min":19.670999999999999},"min":{"1min":7.1609999999999996,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":191.435,"5min":287.95100000000002,"15min":287.95100000000002},"last":11.076000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":13.749000000000001,"5min":25.651,"15min":2144.5120000000002},"min":{"1min":6.8760000000000003,"5min":6.8760000000000003,"15min":6.8760000000000003},"max":{"1min":24.265999999999998,"5min":688.24300000000005,"15min":75975.607000000004},"last":16.321999999999999},{"interface":"front","average":{"1min":14.723000000000001,"5min":27.239999999999998,"15min":2143.9769999999999},"min":{"1min":9.7739999999999991,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.850999999999999,"5min":610.89800000000002,"15min":75971.296000000002},"last":15.746}]},{"osd":7,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":16.059000000000001,"5min":139.43199999999999,"15min":66.170000000000002},"min":{"1min":13.545,"5min":10.411,"15min":10.411},"max":{"1min":18.268000000000001,"5min":611.59799999999996,"15min":611.59799999999996},"last":17.071999999999999},{"interface":"front","average":{"1min":13.574999999999999,"5min":139.542,"15min":57.654000000000003},"min":{"1min":8.984,"5min":8.609,"15min":7.0410000000000004},"max":{"1min":17.788,"5min":629.07500000000005,"15min":629.07500000000005},"last":11.609}]}]},{"osd":3,"up_from":736,"seq":3161095930311,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103980,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267860,"statfs":{"total":96636764160,"available":96530288640,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":14.256,"5min":12.529999999999999,"15min":14.252000000000001},"min":{"1min":4.673,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":79.668000000000006,"5min":101.90600000000001,"15min":159.828},"last":10.055},{"interface":"front","average":{"1min":18.561,"5min":13.141999999999999,"15min":14.512},"min":{"1min":5.7389999999999999,"5min":5.2060000000000004,"15min":4.4000000000000004},"max":{"1min":131.036,"5min":131.036,"15min":153.459},"last":9.4670000000000005}]},{"osd":1,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":19.010000000000002,"5min":13.263,"15min":386.18900000000002},"min":{"1min":7.3339999999999996,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":152.60900000000001,"5min":152.60900000000001,"15min":20852.026000000002},"last":8.2560000000000002},{"interface":"front","average":{"1min":20.308,"5min":15.276999999999999,"15min":382.14400000000001},"min":{"1min":8.9550000000000001,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":93.549999999999997,"5min":93.549999999999997,"15min":20733.266},"last":11.648999999999999}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":15.489000000000001},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":7.069}]},{"osd":4,"last update":"Thu Nov 2 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":25.468,"5min":22.239999999999998,"15min":21.866},"min":{"1min":8.5199999999999996,"5min":8.0069999999999997,"15min":7.0129999999999999},"max":{"1min":202.30099999999999,"5min":202.30099999999999,"15min":202.30099999999999},"last":12.997},{"interface":"front","average":{"1min":25.638999999999999,"5min":21.823,"15min":22.591000000000001},"min":{"1min":7.375,"5min":7.375,"15min":6.944},"max":{"1min":179.37299999999999,"5min":179.37299999999999,"15min":210.239},"last":12.718}]},{"osd":5,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":32.866999999999997,"5min":22.800000000000001,"15min":23.951000000000001},"min":{"1min":10.111000000000001,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":180.82499999999999,"5min":180.82499999999999,"15min":290.07100000000003},"last":8.8070000000000004},{"interface":"front","average":{"1min":28.981000000000002,"5min":19.195,"15min":20.167999999999999},"min":{"1min":8.7330000000000005,"5min":8.4949999999999992,"15min":6.8220000000000001},"max":{"1min":170.89599999999999,"5min":170.89599999999999,"15min":268.577},"last":13.951000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":25.896999999999998,"5min":19.382999999999999,"15min":1643.183},"min":{"1min":12.167,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":102.196,"5min":102.196,"15min":76086.231},"last":14.666},{"interface":"front","average":{"1min":20.253,"5min":19.183,"15min":1650.2560000000001},"min":{"1min":10.005000000000001,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":62.909999999999997,"5min":71.622,"15min":76087.114000000001},"last":19.161999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":30.446000000000002,"5min":20.318999999999999,"15min":23.869},"min":{"1min":12.579000000000001,"5min":10.125,"15min":8.8870000000000005},"max":{"1min":178.74299999999999,"5min":178.74299999999999,"15min":178.74299999999999},"last":16.667000000000002},{"interface":"front","average":{"1min":24.611999999999998,"5min":19.600999999999999,"15min":24.111000000000001},"min":{"1min":10.590999999999999,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":118.634,"5min":118.634,"15min":140.87299999999999},"last":17.809999999999999}]}]},{"osd":7,"up_from":875,"seq":3758096384434,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":12.968999999999999,"5min":12.625999999999999,"15min":15.327999999999999},"min":{"1min":5.4749999999999996,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":87.353999999999999,"5min":87.353999999999999,"15min":87.353999999999999},"last":6.7469999999999999},{"interface":"front","average":{"1min":12.054,"5min":11.366,"15min":17.962},"min":{"1min":5.3380000000000001,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":62.587000000000003,"5min":62.587000000000003,"15min":97.984999999999999},"last":9.4730000000000008}]},{"osd":1,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":14.287000000000001,"5min":14.279,"15min":21.698},"min":{"1min":6.2439999999999998,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":34.357999999999997,"5min":48.886000000000003,"15min":178.56},"last":8.4369999999999994},{"interface":"front","average":{"1min":10.861000000000001,"5min":11.736000000000001,"15min":14.888999999999999},"min":{"1min":6.6029999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":32.017000000000003,"5min":47.533999999999999,"15min":66.081999999999994},"last":7.8650000000000002}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":11.602},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":11.067}]},{"osd":3,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":21.795000000000002,"5min":21.023,"15min":19.797000000000001},"min":{"1min":7.6689999999999996,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":143.935,"5min":253.71000000000001,"15min":253.71000000000001},"last":12.207000000000001},{"interface":"front","average":{"1min":19.591999999999999,"5min":16.908000000000001,"15min":18.48},"min":{"1min":8.8000000000000007,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":76.149000000000001,"5min":101.316,"15min":101.316},"last":10.127000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":14.881,"5min":9823.1419999999998,"15min":3300.5569999999998},"min":{"1min":7.5549999999999997,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":29.866,"5min":66277.987999999998,"15min":66277.987999999998},"last":15.396000000000001},{"interface":"front","average":{"1min":18.414000000000001,"5min":9829.7340000000004,"15min":3300.77},"min":{"1min":9.0220000000000002,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":77.519999999999996,"5min":66301.380000000005,"15min":66301.380000000005},"last":13.487}]},{"osd":5,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":18.657,"5min":20.384,"15min":22.456},"min":{"1min":8.6859999999999999,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":86.191999999999993,"5min":263.65699999999998,"15min":263.65699999999998},"last":12.968999999999999},{"interface":"front","average":{"1min":26.029,"5min":22.896999999999998,"15min":24.760000000000002},"min":{"1min":8.8989999999999991,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":242.703,"5min":242.703,"15min":242.703},"last":14.813000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":19.016999999999999,"5min":23.106000000000002,"15min":25.989999999999998},"min":{"1min":10.065,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":75.099999999999994,"5min":285.56999999999999,"15min":285.56999999999999},"last":16.634},{"interface":"front","average":{"1min":18.521999999999998,"5min":21.626999999999999,"15min":23.535},"min":{"1min":13.172000000000001,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":39.274000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":16.035}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:13:55.149 INFO:tasks.ceph:pgid 10.17 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:55.150 INFO:tasks.ceph:pgid 10.15 last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:55.150 INFO:tasks.ceph:pgid 10.1d last_scrub_stamp 2023-11-02T15:35:07.265583+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=15, tm_min=35, tm_sec=7, tm_wday=3, tm_yday=306, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=2, tm_hour=16, tm_min=13, tm_sec=25, tm_wday=3, tm_yday=306, tm_isdst=0) 2023-11-02T16:13:55.151 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-02T16:13:59.511 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:11.664 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.513 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:11.667 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.536 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:11.689 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.563 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:11.660 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.567 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:11.664 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.578 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:11.731 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.581 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:11.735 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.582 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:11.678 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:13:59.595 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:11.749 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.530 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:21.683 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.535 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:21.689 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.551 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:21.705 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.588 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:21.684 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.589 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:21.743 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.590 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:21.686 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.593 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:21.746 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.605 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:21.701 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:09.605 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:21.758 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:15.152 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-02T16:14:15.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.251+0000 7f4163788640 10 monclient: get_monmap_and_config 2023-11-02T16:14:15.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.251+0000 7f4163788640 10 monclient: build_initial_monmap 2023-11-02T16:14:15.253 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.251+0000 7f4163788640 10 monclient: monmap: 2023-11-02T16:14:15.254 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:14:15.258 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.259 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:14:15.252534+0000 2023-11-02T16:14:15.259 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:14:15.252534+0000 2023-11-02T16:14:15.259 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:14:15.259 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.259 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:14:15.260 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:14:15.260 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:14:15.260 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:14:15.260 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:14:15.260 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 1 Processor -- start 2023-11-02T16:14:15.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 1 -- start start 2023-11-02T16:14:15.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 10 monclient: init 2023-11-02T16:14:15.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 10 monclient: _reopen_session rank -1 2023-11-02T16:14:15.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2023-11-02T16:14:15.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.252+0000 7f4163788640 10 monclient(hunting): picked mon.noname-b con 0x7f415c0b4260 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:14:15.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f415c141730 addr v1:172.21.15.175:6789/0 2023-11-02T16:14:15.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f415c13e520 0x7f415c13e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): picked mon.noname-c con 0x7f415c13e520 addr v2:172.21.15.175:3300/0 2023-11-02T16:14:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0a9210 con 0x7f415c141730 2023-11-02T16:14:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c0aa550 con 0x7f415c13e520 2023-11-02T16:14:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0b4640 con 0x7f415c0b4260 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): _renew_subs 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4163788640 10 monclient(hunting): authenticate will time out at 3782.594720s 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.253+0000 7f4161524640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38212/0 (socket says 172.21.15.174:38212) 2023-11-02T16:14:15.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 1 -- 172.21.15.174:0/294667744 learned_addr learned my addr 172.21.15.174:0/294667744 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:14:15.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): get_auth_request con 0x7f415c0b4260 auth_method 0 2023-11-02T16:14:15.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.268 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161d25640 1 --2- 172.21.15.174:0/294667744 >> v2:172.21.15.175:3300/0 conn(0x7f415c13e520 0x7f415c13e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161d25640 10 monclient(hunting): get_auth_request con 0x7f415c13e520 auth_method 0 2023-11-02T16:14:15.269 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161d25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161d25640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.254+0000 7f4161d25640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.255+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.255+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.255+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 10 monclient(hunting): handle_auth_done global_id 16973 payload 274 2023-11-02T16:14:15.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 1 -- 172.21.15.174:0/294667744 >> v2:172.21.15.175:3300/0 conn(0x7f415c13e520 msgr2=0x7f415c13e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:14:15.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 1 --2- 172.21.15.174:0/294667744 >> v2:172.21.15.175:3300/0 conn(0x7f415c13e520 0x7f415c13e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 1 -- 172.21.15.174:0/294667744 >> v1:172.21.15.175:6789/0 conn(0x7f415c141730 legacy=0x7f415c141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 10 monclient: _finish_hunting 0 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 1 monclient: found mon.noname-b 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.174:3301/0 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.256+0000 7f4161524640 1 -- 172.21.15.174:0/294667744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c142280 con 0x7f415c0b4260 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.258+0000 7f4161524640 1 --2- 172.21.15.174:0/294667744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f56ceec1afc4eab server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:15.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 1 -- 172.21.15.174:0/294667744 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158031640 con 0x7f415c0b4260 2023-11-02T16:14:15.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:14:15.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: dump: 2023-11-02T16:14:15.273 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.273 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.274 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.274 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.274 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.274 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.274 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: _finish_auth 0 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: _check_auth_tickets 2023-11-02T16:14:15.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:14:15.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 1 -- 172.21.15.174:0/294667744 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41580317a0 con 0x7f415c0b4260 2023-11-02T16:14:15.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4163788640 5 monclient: authenticate success, global_id 16973 2023-11-02T16:14:15.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4163788640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:14:15.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:14:15.276 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4163788640 10 monclient: get_monmap_and_config success 2023-11-02T16:14:15.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 1 -- 172.21.15.174:0/294667744 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158031b30 con 0x7f415c0b4260 2023-11-02T16:14:15.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4163788640 4 set_mon_vals no callback set 2023-11-02T16:14:15.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:14:15.277 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.261+0000 7f4153fff640 10 monclient: dump: 2023-11-02T16:14:15.278 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.278 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.278 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.278 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.278 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 10 monclient: shutdown 2023-11-02T16:14:15.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:14:15.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 -- 172.21.15.174:0/294667744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 msgr2=0x7f415c0a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 --2- 172.21.15.174:0/294667744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415802b8f0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 -- 172.21.15.174:0/294667744 shutdown_connections 2023-11-02T16:14:15.280 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 --2- 172.21.15.174:0/294667744 >> v2:172.21.15.175:3300/0 conn(0x7f415c13e520 0x7f415c13e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 --2- 172.21.15.174:0/294667744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c0b4260 0x7f415c0a8870 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 -- 172.21.15.174:0/294667744 >> 172.21.15.174:0/294667744 conn(0x7f415c065d50 msgr2=0x7f415c066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:14:15.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 -- 172.21.15.174:0/294667744 shutdown_connections 2023-11-02T16:14:15.281 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.262+0000 7f4163788640 1 -- 172.21.15.174:0/294667744 wait complete. 2023-11-02T16:14:15.285 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 10 monclient: build_initial_monmap 2023-11-02T16:14:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 10 monclient: monmap: 2023-11-02T16:14:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:14:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:14:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:14:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:14:15.287 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:14:15.287 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.287 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:14:15.287 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:14:15.287 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 librados: starting msgr at 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 librados: starting objecter 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 Processor -- start 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 -- start start 2023-11-02T16:14:15.288 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 librados: setting wanted keys 2023-11-02T16:14:15.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 1 librados: calling monclient init 2023-11-02T16:14:15.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.263+0000 7f4163788640 10 monclient: init 2023-11-02T16:14:15.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient: _reopen_session rank -1 2023-11-02T16:14:15.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient: _add_conns ranks=[1,2,0] 2023-11-02T16:14:15.289 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f415c0b4260 0x7f415c1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): picked mon.noname-b con 0x7f415c0b4260 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:14:15.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): picked mon.noname-c con 0x7f415c13e520 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:14:15.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f415c141730 0x7f415c1d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): picked mon.noname-a con 0x7f415c141730 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:14:15.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c13b3b0 con 0x7f415c141730 2023-11-02T16:14:15.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c13b070 con 0x7f415c0b4260 2023-11-02T16:14:15.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c13a070 con 0x7f415c13e520 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): _renew_subs 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4163788640 10 monclient(hunting): authenticate will time out at 3782.605971s 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4160d23640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4161d25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f415c141730 0x7f415c1d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.292 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.264+0000 7f4161524640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f415c0b4260 0x7f415c1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:38228/0 (socket says 172.21.15.174:38228) 2023-11-02T16:14:15.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 1 -- 172.21.15.174:0/1385128571 learned_addr learned my addr 172.21.15.174:0/1385128571 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:14:15.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): get_auth_request con 0x7f415c13e520 auth_method 0 2023-11-02T16:14:15.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.293 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): get_auth_request con 0x7f415c141730 auth_method 0 2023-11-02T16:14:15.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.294 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): get_auth_request con 0x7f415c0b4260 auth_method 0 2023-11-02T16:14:15.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.295 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4160d23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161d25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.265+0000 7f4161524640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 10 monclient(hunting): handle_auth_done global_id 16976 payload 824 2023-11-02T16:14:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 -- 172.21.15.174:0/1385128571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f415c0b4260 msgr2=0x7f415c1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f415c0b4260 0x7f415c1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 -- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f415c141730 msgr2=0x7f415c1d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f415c141730 0x7f415c1d67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 10 monclient: _finish_hunting 0 2023-11-02T16:14:15.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 monclient: found mon.noname-c 2023-11-02T16:14:15.298 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:14:15.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.174:3301/0 2023-11-02T16:14:15.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.268+0000 7f4160d23640 1 -- 172.21.15.174:0/1385128571 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580094d0 con 0x7f415c13e520 2023-11-02T16:14:15.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.270+0000 7f4160d23640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c3c0 tx=0x7f414c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:15.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.270+0000 7f4151ffb640 10 client.?.objecter ms_handle_connect 0x7f415c13e520 2023-11-02T16:14:15.299 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.270+0000 7f4151ffb640 10 client.?.objecter resend_mon_ops 2023-11-02T16:14:15.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0052d0 con 0x7f415c13e520 2023-11-02T16:14:15.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:14:15.300 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: dump: 2023-11-02T16:14:15.300 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.301 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.302 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.302 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.302 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: _finish_auth 0 2023-11-02T16:14:15.302 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: _check_auth_tickets 2023-11-02T16:14:15.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:14:15.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c005450 con 0x7f415c13e520 2023-11-02T16:14:15.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4163788640 5 monclient: authenticate success, global_id 16976 2023-11-02T16:14:15.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4163788640 10 monclient: _renew_subs 2023-11-02T16:14:15.303 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4163788640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:14:15.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c13a2a0 con 0x7f415c13e520 2023-11-02T16:14:15.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:14:15.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c007260 con 0x7f415c13e520 2023-11-02T16:14:15.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.304 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-11-02T16:14:15.305 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f4151ffb640 10 monclient: dump: 2023-11-02T16:14:15.305 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.305 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.305 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.305 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.306 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.306 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.306 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.306 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.306 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.273+0000 7f41537fe640 4 set_mon_vals no callback set 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.274+0000 7f4163788640 10 client.16976.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.274+0000 7f4163788640 10 monclient: _renew_subs 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.274+0000 7f4163788640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:14:15.307 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.274+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c1d6ee0 con 0x7f415c13e520 2023-11-02T16:14:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.274+0000 7f4163788640 1 librados: init done 2023-11-02T16:14:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.275+0000 7f4163788640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.275+0000 7f4163788640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.275+0000 7f4163788640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.174:3301/0 2023-11-02T16:14:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.275+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c0aa7d0 con 0x7f415c13e520 2023-11-02T16:14:15.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.292+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f414c0073c0 con 0x7f415c13e520 2023-11-02T16:14:15.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.293+0000 7f4151ffb640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 0x7f414006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.293+0000 7f4161524640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 0x7f414006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.293+0000 7f4161524640 10 monclient: get_auth_request con 0x7f414006be10 auth_method 0 2023-11-02T16:14:15.310 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.293+0000 7f4151ffb640 10 client.16976.objecter ms_handle_connect 0x7f414006be10 2023-11-02T16:14:15.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.293+0000 7f4161524640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 0x7f414006e2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:15.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.295+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f414c057440 con 0x7f415c13e520 2023-11-02T16:14:15.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.295+0000 7f4151ffb640 10 client.16976.objecter ms_dispatch 0x7f415c000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:14:15.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.295+0000 7f4151ffb640 3 client.16976.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:14:15.311 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.295+0000 7f4151ffb640 3 client.16976.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:14:15.312 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.295+0000 7f4151ffb640 20 client.16976.objecter dump_active .. 0 homeless 2023-11-02T16:14:15.540 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.539+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f414c053780 con 0x7f415c13e520 2023-11-02T16:14:15.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.539+0000 7f4151ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:15.541 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.539+0000 7f4151ffb640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:14:15.721 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.719+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 --> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f415c067920 con 0x7f414006be10 2023-11-02T16:14:15.726 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.724+0000 7f4151ffb640 1 -- 172.21.15.174:0/1385128571 <== mgr.4107 v2:172.21.15.174:6800/85844 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152274 (secure 0 0 0) 0x7f415c067920 con 0x7f414006be10 2023-11-02T16:14:15.728 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-02T16:14:15.729 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2023-11-02T16:14:15.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 10 librados: watch_flush enter 2023-11-02T16:14:15.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 10 librados: watch_flush exit 2023-11-02T16:14:15.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 20 client.16976.objecter shutdown clearing up homeless session... 2023-11-02T16:14:15.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 10 client.16976.objecter successfully canceled tick 2023-11-02T16:14:15.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 msgr2=0x7f414006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.726+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 0x7f414006e2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415803c040 comp rx=0 tx=0).stop 2023-11-02T16:14:15.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 10 monclient: shutdown 2023-11-02T16:14:15.732 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:14:15.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 msgr2=0x7f415c13ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c3c0 tx=0x7f414c00c880 comp rx=0 tx=0).stop 2023-11-02T16:14:15.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 shutdown_connections 2023-11-02T16:14:15.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f415c141730 0x7f415c1d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7f414006be10 0x7f414006e2d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f415c13e520 0x7f415c13ab30 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 --2- 172.21.15.174:0/1385128571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f415c0b4260 0x7f415c1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 >> 172.21.15.174:0/1385128571 conn(0x7f415c065d50 msgr2=0x7f415c0b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:14:15.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 shutdown_connections 2023-11-02T16:14:15.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 -- 172.21.15.174:0/1385128571 wait complete. 2023-11-02T16:14:15.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.727+0000 7f4163788640 1 librados: shutdown 2023-11-02T16:14:15.739 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":2245,"stamp":"2023-11-02T16:14:15.418407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":609908,"kb_used_data":86068,"kb_used_omap":0,"kb_used_meta":523776,"kb_avail":754364812,"statfs":{"total":773094113280,"available":772469567488,"internally_reserved":0,"allocated":88133632,"data_stored":51451729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":536346624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003306"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:42.051930+0000","last_change":"2023-11-02T16:13:42.051930+0000","last_active":"2023-11-02T16:13:42.051930+0000","last_peered":"2023-11-02T16:13:42.051930+0000","last_clean":"2023-11-02T16:13:42.051930+0000","last_became_active":"2023-11-02T16:13:17.151525+0000","last_became_peered":"2023-11-02T16:13:17.151525+0000","last_unstale":"2023-11-02T16:13:42.051930+0000","last_undegraded":"2023-11-02T16:13:42.051930+0000","last_fullsized":"2023-11-02T16:13:42.051930+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:42.049538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:42.049538+0000","last_clean_scrub_stamp":"2023-11-02T16:13:42.049538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:21:40.305486+0000","scrub_duration":0.039569602000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1025,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:54.957865+0000","last_change":"2023-11-02T16:13:54.962357+0000","last_active":"2023-11-02T16:13:54.957865+0000","last_peered":"2023-11-02T16:13:54.957865+0000","last_clean":"2023-11-02T16:13:54.957865+0000","last_became_active":"2023-11-02T16:07:27.300929+0000","last_became_peered":"2023-11-02T16:07:27.300929+0000","last_unstale":"2023-11-02T16:13:54.957865+0000","last_undegraded":"2023-11-02T16:13:54.957865+0000","last_fullsized":"2023-11-02T16:13:54.957865+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:54.957131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:54.957131+0000","last_clean_scrub_stamp":"2023-11-02T16:13:54.957131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:57:26.777169+0000","scrub_duration":0.066366169000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":947,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:47.229021+0000","last_change":"2023-11-02T16:13:47.230619+0000","last_active":"2023-11-02T16:13:47.229021+0000","last_peered":"2023-11-02T16:13:47.229021+0000","last_clean":"2023-11-02T16:13:47.229021+0000","last_became_active":"2023-11-02T16:07:27.389056+0000","last_became_peered":"2023-11-02T16:07:27.389056+0000","last_unstale":"2023-11-02T16:13:47.229021+0000","last_undegraded":"2023-11-02T16:13:47.229021+0000","last_fullsized":"2023-11-02T16:13:47.229021+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:47.228308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:47.228308+0000","last_clean_scrub_stamp":"2023-11-02T16:13:47.228308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:01:59.650918+0000","scrub_duration":0.040728582999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":988,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:51.055432+0000","last_change":"2023-11-02T16:13:51.055432+0000","last_active":"2023-11-02T16:13:51.055432+0000","last_peered":"2023-11-02T16:13:51.055432+0000","last_clean":"2023-11-02T16:13:51.055432+0000","last_became_active":"2023-11-02T15:35:17.336217+0000","last_became_peered":"2023-11-02T15:35:17.336217+0000","last_unstale":"2023-11-02T16:13:51.055432+0000","last_undegraded":"2023-11-02T16:13:51.055432+0000","last_fullsized":"2023-11-02T16:13:51.055432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:51.053016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:51.053016+0000","last_clean_scrub_stamp":"2023-11-02T16:13:51.053016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:29:24.405918+0000","scrub_duration":0.041325683000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":986,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.175163+0000","last_change":"2023-11-02T16:13:41.176774+0000","last_active":"2023-11-02T16:13:41.175163+0000","last_peered":"2023-11-02T16:13:41.175163+0000","last_clean":"2023-11-02T16:13:41.175163+0000","last_became_active":"2023-11-02T15:35:14.201056+0000","last_became_peered":"2023-11-02T15:35:14.201056+0000","last_unstale":"2023-11-02T16:13:41.175163+0000","last_undegraded":"2023-11-02T16:13:41.175163+0000","last_fullsized":"2023-11-02T16:13:41.175163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.174422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.174422+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.174422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:14:12.557347+0000","scrub_duration":0.046185103999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":934,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:37.163987+0000","last_change":"2023-11-02T16:13:37.168167+0000","last_active":"2023-11-02T16:13:37.163987+0000","last_peered":"2023-11-02T16:13:37.163987+0000","last_clean":"2023-11-02T16:13:37.163987+0000","last_became_active":"2023-11-02T15:59:17.693851+0000","last_became_peered":"2023-11-02T15:59:17.693851+0000","last_unstale":"2023-11-02T16:13:37.163987+0000","last_undegraded":"2023-11-02T16:13:37.163987+0000","last_fullsized":"2023-11-02T16:13:37.163987+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:37.163062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:37.163062+0000","last_clean_scrub_stamp":"2023-11-02T16:13:37.163062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:04:26.268836+0000","scrub_duration":0.065834622999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1012,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.116761+0000","last_change":"2023-11-02T16:13:44.118515+0000","last_active":"2023-11-02T16:13:44.116761+0000","last_peered":"2023-11-02T16:13:44.116761+0000","last_clean":"2023-11-02T16:13:44.116761+0000","last_became_active":"2023-11-02T16:13:16.814387+0000","last_became_peered":"2023-11-02T16:13:16.814387+0000","last_unstale":"2023-11-02T16:13:44.116761+0000","last_undegraded":"2023-11-02T16:13:44.116761+0000","last_fullsized":"2023-11-02T16:13:44.116761+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.116015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.116015+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.116015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:46:01.480651+0000","scrub_duration":0.040346910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":986,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.462604+0000","last_change":"2023-11-02T16:13:32.468370+0000","last_active":"2023-11-02T16:13:32.462604+0000","last_peered":"2023-11-02T16:13:32.462604+0000","last_clean":"2023-11-02T16:13:32.462604+0000","last_became_active":"2023-11-02T15:35:30.065096+0000","last_became_peered":"2023-11-02T15:35:30.065096+0000","last_unstale":"2023-11-02T16:13:32.462604+0000","last_undegraded":"2023-11-02T16:13:32.462604+0000","last_fullsized":"2023-11-02T16:13:32.462604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.461412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.461412+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.461412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:17:47.255889+0000","scrub_duration":0.072050043999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":975,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:38.012415+0000","last_change":"2023-11-02T16:13:38.014259+0000","last_active":"2023-11-02T16:13:38.012415+0000","last_peered":"2023-11-02T16:13:38.012415+0000","last_clean":"2023-11-02T16:13:38.012415+0000","last_became_active":"2023-11-02T16:07:27.402041+0000","last_became_peered":"2023-11-02T16:07:27.402041+0000","last_unstale":"2023-11-02T16:13:38.012415+0000","last_undegraded":"2023-11-02T16:13:38.012415+0000","last_fullsized":"2023-11-02T16:13:38.012415+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:38.011629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:38.011629+0000","last_clean_scrub_stamp":"2023-11-02T16:13:38.011629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:37:20.779523+0000","scrub_duration":0.071776143000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:42.254430+0000","last_change":"2023-11-02T16:13:42.256054+0000","last_active":"2023-11-02T16:13:42.254430+0000","last_peered":"2023-11-02T16:13:42.254430+0000","last_clean":"2023-11-02T16:13:42.254430+0000","last_became_active":"2023-11-02T16:13:16.879104+0000","last_became_peered":"2023-11-02T16:13:16.879104+0000","last_unstale":"2023-11-02T16:13:42.254430+0000","last_undegraded":"2023-11-02T16:13:42.254430+0000","last_fullsized":"2023-11-02T16:13:42.254430+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:42.253698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:42.253698+0000","last_clean_scrub_stamp":"2023-11-02T16:13:42.253698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:07:54.847830+0000","scrub_duration":0.050904000999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1026,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.240445+0000","last_change":"2023-11-02T16:13:41.240445+0000","last_active":"2023-11-02T16:13:41.240445+0000","last_peered":"2023-11-02T16:13:41.240445+0000","last_clean":"2023-11-02T16:13:41.240445+0000","last_became_active":"2023-11-02T16:13:16.904158+0000","last_became_peered":"2023-11-02T16:13:16.904158+0000","last_unstale":"2023-11-02T16:13:41.240445+0000","last_undegraded":"2023-11-02T16:13:41.240445+0000","last_fullsized":"2023-11-02T16:13:41.240445+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.234782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.234782+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.234782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:00:16.955119+0000","scrub_duration":0.037739861,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1025,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:55.907125+0000","last_change":"2023-11-02T16:13:55.907125+0000","last_active":"2023-11-02T16:13:55.907125+0000","last_peered":"2023-11-02T16:13:55.907125+0000","last_clean":"2023-11-02T16:13:55.907125+0000","last_became_active":"2023-11-02T16:13:17.184008+0000","last_became_peered":"2023-11-02T16:13:17.184008+0000","last_unstale":"2023-11-02T16:13:55.907125+0000","last_undegraded":"2023-11-02T16:13:55.907125+0000","last_fullsized":"2023-11-02T16:13:55.907125+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:55.901854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:55.901854+0000","last_clean_scrub_stamp":"2023-11-02T16:13:55.901854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:53:32.426807+0000","scrub_duration":0.03414147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":999,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:45.912186+0000","last_change":"2023-11-02T16:13:45.913860+0000","last_active":"2023-11-02T16:13:45.912186+0000","last_peered":"2023-11-02T16:13:45.912186+0000","last_clean":"2023-11-02T16:13:45.912186+0000","last_became_active":"2023-11-02T15:59:17.628415+0000","last_became_peered":"2023-11-02T15:59:17.628415+0000","last_unstale":"2023-11-02T16:13:45.912186+0000","last_undegraded":"2023-11-02T16:13:45.912186+0000","last_fullsized":"2023-11-02T16:13:45.912186+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:45.911467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:45.911467+0000","last_clean_scrub_stamp":"2023-11-02T16:13:45.911467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:26:44.521107+0000","scrub_duration":0.066455971000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:43.947682+0000","last_change":"2023-11-02T16:13:43.952967+0000","last_active":"2023-11-02T16:13:43.947682+0000","last_peered":"2023-11-02T16:13:43.947682+0000","last_clean":"2023-11-02T16:13:43.947682+0000","last_became_active":"2023-11-02T15:59:17.572288+0000","last_became_peered":"2023-11-02T15:59:17.572288+0000","last_unstale":"2023-11-02T16:13:43.947682+0000","last_undegraded":"2023-11-02T16:13:43.947682+0000","last_fullsized":"2023-11-02T16:13:43.947682+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:43.946966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:43.946966+0000","last_clean_scrub_stamp":"2023-11-02T16:13:43.946966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:34:12.457157+0000","scrub_duration":0.033136158999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":990,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.062291+0000","last_change":"2023-11-02T16:13:41.067307+0000","last_active":"2023-11-02T16:13:41.062291+0000","last_peered":"2023-11-02T16:13:41.062291+0000","last_clean":"2023-11-02T16:13:41.062291+0000","last_became_active":"2023-11-02T15:35:23.793161+0000","last_became_peered":"2023-11-02T15:35:23.793161+0000","last_unstale":"2023-11-02T16:13:41.062291+0000","last_undegraded":"2023-11-02T16:13:41.062291+0000","last_fullsized":"2023-11-02T16:13:41.062291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.061561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.061561+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.061561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:09:22.648006+0000","scrub_duration":0.047085142000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":994,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.930206+0000","last_change":"2023-11-02T16:13:32.930206+0000","last_active":"2023-11-02T16:13:32.930206+0000","last_peered":"2023-11-02T16:13:32.930206+0000","last_clean":"2023-11-02T16:13:32.930206+0000","last_became_active":"2023-11-02T15:34:01.982172+0000","last_became_peered":"2023-11-02T15:34:01.982172+0000","last_unstale":"2023-11-02T16:13:32.930206+0000","last_undegraded":"2023-11-02T16:13:32.930206+0000","last_fullsized":"2023-11-02T16:13:32.930206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.927710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.927710+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.927710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:05:46.839406+0000","scrub_duration":0.045056061000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1004,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:43.111816+0000","last_change":"2023-11-02T16:13:43.113486+0000","last_active":"2023-11-02T16:13:43.111816+0000","last_peered":"2023-11-02T16:13:43.111816+0000","last_clean":"2023-11-02T16:13:43.111816+0000","last_became_active":"2023-11-02T15:59:17.875496+0000","last_became_peered":"2023-11-02T15:59:17.875496+0000","last_unstale":"2023-11-02T16:13:43.111816+0000","last_undegraded":"2023-11-02T16:13:43.111816+0000","last_fullsized":"2023-11-02T16:13:43.111816+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:43.111090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:43.111090+0000","last_clean_scrub_stamp":"2023-11-02T16:13:43.111090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:51:40.525837+0000","scrub_duration":0.057244171000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1006,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:41.936357+0000","last_change":"2023-11-02T16:13:41.936357+0000","last_active":"2023-11-02T16:13:41.936357+0000","last_peered":"2023-11-02T16:13:41.936357+0000","last_clean":"2023-11-02T16:13:41.936357+0000","last_became_active":"2023-11-02T15:59:17.603695+0000","last_became_peered":"2023-11-02T15:59:17.603695+0000","last_unstale":"2023-11-02T16:13:41.936357+0000","last_undegraded":"2023-11-02T16:13:41.936357+0000","last_fullsized":"2023-11-02T16:13:41.936357+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:41.933984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:41.933984+0000","last_clean_scrub_stamp":"2023-11-02T16:13:41.933984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:02:39.382548+0000","scrub_duration":0.028975570999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1028,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:38.283506+0000","last_change":"2023-11-02T16:13:38.285096+0000","last_active":"2023-11-02T16:13:38.283506+0000","last_peered":"2023-11-02T16:13:38.283506+0000","last_clean":"2023-11-02T16:13:38.283506+0000","last_became_active":"2023-11-02T16:13:17.145406+0000","last_became_peered":"2023-11-02T16:13:17.145406+0000","last_unstale":"2023-11-02T16:13:38.283506+0000","last_undegraded":"2023-11-02T16:13:38.283506+0000","last_fullsized":"2023-11-02T16:13:38.283506+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:38.282778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:38.282778+0000","last_clean_scrub_stamp":"2023-11-02T16:13:38.282778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T01:47:12.523508+0000","scrub_duration":0.027973423000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":967,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:32.986922+0000","last_change":"2023-11-02T16:13:32.986922+0000","last_active":"2023-11-02T16:13:32.986922+0000","last_peered":"2023-11-02T16:13:32.986922+0000","last_clean":"2023-11-02T16:13:32.986922+0000","last_became_active":"2023-11-02T15:43:56.969748+0000","last_became_peered":"2023-11-02T15:43:56.969748+0000","last_unstale":"2023-11-02T16:13:32.986922+0000","last_undegraded":"2023-11-02T16:13:32.986922+0000","last_fullsized":"2023-11-02T16:13:32.986922+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:32.984352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:32.984352+0000","last_clean_scrub_stamp":"2023-11-02T16:13:32.984352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:23:32.900645+0000","scrub_duration":0.034194417999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":547,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:40.142587+0000","last_change":"2023-11-02T16:13:40.142587+0000","last_active":"2023-11-02T16:13:40.142587+0000","last_peered":"2023-11-02T16:13:40.142587+0000","last_clean":"2023-11-02T16:13:40.142587+0000","last_became_active":"2023-11-02T16:07:27.480747+0000","last_became_peered":"2023-11-02T16:07:27.480747+0000","last_unstale":"2023-11-02T16:13:40.142587+0000","last_undegraded":"2023-11-02T16:13:40.142587+0000","last_fullsized":"2023-11-02T16:13:40.142587+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:40.140073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:40.140073+0000","last_clean_scrub_stamp":"2023-11-02T16:13:40.140073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:34:40.870971+0000","scrub_duration":0.046347369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1021,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:31.076061+0000","last_change":"2023-11-02T16:13:31.076061+0000","last_active":"2023-11-02T16:13:31.076061+0000","last_peered":"2023-11-02T16:13:31.076061+0000","last_clean":"2023-11-02T16:13:31.076061+0000","last_became_active":"2023-11-02T15:59:17.523154+0000","last_became_peered":"2023-11-02T15:59:17.523154+0000","last_unstale":"2023-11-02T16:13:31.076061+0000","last_undegraded":"2023-11-02T16:13:31.076061+0000","last_fullsized":"2023-11-02T16:13:31.076061+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:31.073530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:31.073530+0000","last_clean_scrub_stamp":"2023-11-02T16:13:31.073530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T18:28:08.702885+0000","scrub_duration":0.03585907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":992,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.158824+0000","last_change":"2023-11-02T16:13:34.158824+0000","last_active":"2023-11-02T16:13:34.158824+0000","last_peered":"2023-11-02T16:13:34.158824+0000","last_clean":"2023-11-02T16:13:34.158824+0000","last_became_active":"2023-11-02T15:35:16.426770+0000","last_became_peered":"2023-11-02T15:35:16.426770+0000","last_unstale":"2023-11-02T16:13:34.158824+0000","last_undegraded":"2023-11-02T16:13:34.158824+0000","last_fullsized":"2023-11-02T16:13:34.158824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.152960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.152960+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.152960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:04:12.998703+0000","scrub_duration":0.086841235000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:39.176671+0000","last_change":"2023-11-02T16:13:39.176671+0000","last_active":"2023-11-02T16:13:39.176671+0000","last_peered":"2023-11-02T16:13:39.176671+0000","last_clean":"2023-11-02T16:13:39.176671+0000","last_became_active":"2023-11-02T16:07:27.301714+0000","last_became_peered":"2023-11-02T16:07:27.301714+0000","last_unstale":"2023-11-02T16:13:39.176671+0000","last_undegraded":"2023-11-02T16:13:39.176671+0000","last_fullsized":"2023-11-02T16:13:39.176671+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:39.172970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:39.172970+0000","last_clean_scrub_stamp":"2023-11-02T16:13:39.172970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T20:43:44.770192+0000","scrub_duration":0.066176944000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":949,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.991167+0000","last_change":"2023-11-02T16:13:44.993038+0000","last_active":"2023-11-02T16:13:44.991167+0000","last_peered":"2023-11-02T16:13:44.991167+0000","last_clean":"2023-11-02T16:13:44.991167+0000","last_became_active":"2023-11-02T15:35:10.698873+0000","last_became_peered":"2023-11-02T15:35:10.698873+0000","last_unstale":"2023-11-02T16:13:44.991167+0000","last_undegraded":"2023-11-02T16:13:44.991167+0000","last_fullsized":"2023-11-02T16:13:44.991167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.990415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.990415+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.990415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T22:49:40.351558+0000","scrub_duration":0.036013218999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1006,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:30.029645+0000","last_change":"2023-11-02T16:13:30.031726+0000","last_active":"2023-11-02T16:13:30.029645+0000","last_peered":"2023-11-02T16:13:30.029645+0000","last_clean":"2023-11-02T16:13:30.029645+0000","last_became_active":"2023-11-02T15:59:17.402049+0000","last_became_peered":"2023-11-02T15:59:17.402049+0000","last_unstale":"2023-11-02T16:13:30.029645+0000","last_undegraded":"2023-11-02T16:13:30.029645+0000","last_fullsized":"2023-11-02T16:13:30.029645+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:30.028859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:30.028859+0000","last_clean_scrub_stamp":"2023-11-02T16:13:30.028859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:23:54.699449+0000","scrub_duration":0.03822644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":987,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:49.235869+0000","last_change":"2023-11-02T16:13:49.237577+0000","last_active":"2023-11-02T16:13:49.235869+0000","last_peered":"2023-11-02T16:13:49.235869+0000","last_clean":"2023-11-02T16:13:49.235869+0000","last_became_active":"2023-11-02T15:35:30.079023+0000","last_became_peered":"2023-11-02T15:35:30.079023+0000","last_unstale":"2023-11-02T16:13:49.235869+0000","last_undegraded":"2023-11-02T16:13:49.235869+0000","last_fullsized":"2023-11-02T16:13:49.235869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:49.235124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:49.235124+0000","last_clean_scrub_stamp":"2023-11-02T16:13:49.235124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:05:36.160979+0000","scrub_duration":0.059997438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1032,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.063784+0000","last_change":"2023-11-02T16:13:34.063784+0000","last_active":"2023-11-02T16:13:34.063784+0000","last_peered":"2023-11-02T16:13:34.063784+0000","last_clean":"2023-11-02T16:13:34.063784+0000","last_became_active":"2023-11-02T16:13:17.210864+0000","last_became_peered":"2023-11-02T16:13:17.210864+0000","last_unstale":"2023-11-02T16:13:34.063784+0000","last_undegraded":"2023-11-02T16:13:34.063784+0000","last_fullsized":"2023-11-02T16:13:34.063784+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.060576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.060576+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.060576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:43:43.380598+0000","scrub_duration":0.031708978999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"1030'78","reported_seq":1101,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:30.530233+0000","last_change":"2023-11-02T16:13:30.536191+0000","last_active":"2023-11-02T16:13:30.530233+0000","last_peered":"2023-11-02T16:13:30.530233+0000","last_clean":"2023-11-02T16:13:30.530233+0000","last_became_active":"2023-11-02T15:33:55.660936+0000","last_became_peered":"2023-11-02T15:33:55.660936+0000","last_unstale":"2023-11-02T16:13:30.530233+0000","last_undegraded":"2023-11-02T16:13:30.530233+0000","last_fullsized":"2023-11-02T16:13:30.530233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'78","last_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_deep_scrub":"1030'78","last_deep_scrub_stamp":"2023-11-02T16:13:30.528744+0000","last_clean_scrub_stamp":"2023-11-02T16:13:30.528744+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:03:01.384002+0000","scrub_duration":0.116374897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"1030'8","reported_seq":1032,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:29.185182+0000","last_change":"2023-11-02T16:13:29.187351+0000","last_active":"2023-11-02T16:13:29.185182+0000","last_peered":"2023-11-02T16:13:29.185182+0000","last_clean":"2023-11-02T16:13:29.185182+0000","last_became_active":"2023-11-02T16:13:17.021127+0000","last_became_peered":"2023-11-02T16:13:17.021127+0000","last_unstale":"2023-11-02T16:13:29.185182+0000","last_undegraded":"2023-11-02T16:13:29.185182+0000","last_fullsized":"2023-11-02T16:13:29.185182+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"1030'8","last_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_deep_scrub":"1030'8","last_deep_scrub_stamp":"2023-11-02T16:13:29.183095+0000","last_clean_scrub_stamp":"2023-11-02T16:13:29.183095+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:20:07.879623+0000","scrub_duration":0.12634673699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1016,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:36.223251+0000","last_change":"2023-11-02T16:13:36.229119+0000","last_active":"2023-11-02T16:13:36.223251+0000","last_peered":"2023-11-02T16:13:36.223251+0000","last_clean":"2023-11-02T16:13:36.223251+0000","last_became_active":"2023-11-02T16:13:16.998070+0000","last_became_peered":"2023-11-02T16:13:16.998070+0000","last_unstale":"2023-11-02T16:13:36.223251+0000","last_undegraded":"2023-11-02T16:13:36.223251+0000","last_fullsized":"2023-11-02T16:13:36.223251+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:36.221991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:36.221991+0000","last_clean_scrub_stamp":"2023-11-02T16:13:36.221991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T23:12:00.805971+0000","scrub_duration":0.038230550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1018,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:36.917516+0000","last_change":"2023-11-02T16:13:36.917516+0000","last_active":"2023-11-02T16:13:36.917516+0000","last_peered":"2023-11-02T16:13:36.917516+0000","last_clean":"2023-11-02T16:13:36.917516+0000","last_became_active":"2023-11-02T16:13:16.942001+0000","last_became_peered":"2023-11-02T16:13:16.942001+0000","last_unstale":"2023-11-02T16:13:36.917516+0000","last_undegraded":"2023-11-02T16:13:36.917516+0000","last_fullsized":"2023-11-02T16:13:36.917516+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:36.915123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:36.915123+0000","last_clean_scrub_stamp":"2023-11-02T16:13:36.915123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:04:42.967961+0000","scrub_duration":0.036527178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":976,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.998609+0000","last_change":"2023-11-02T16:13:35.000385+0000","last_active":"2023-11-02T16:13:34.998609+0000","last_peered":"2023-11-02T16:13:34.998609+0000","last_clean":"2023-11-02T16:13:34.998609+0000","last_became_active":"2023-11-02T16:13:17.154584+0000","last_became_peered":"2023-11-02T16:13:17.154584+0000","last_unstale":"2023-11-02T16:13:34.998609+0000","last_undegraded":"2023-11-02T16:13:34.998609+0000","last_fullsized":"2023-11-02T16:13:34.998609+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.997831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.997831+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.997831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T02:37:45.627987+0000","scrub_duration":0.036573472000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1003,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:48.106734+0000","last_change":"2023-11-02T16:13:48.113388+0000","last_active":"2023-11-02T16:13:48.106734+0000","last_peered":"2023-11-02T16:13:48.106734+0000","last_clean":"2023-11-02T16:13:48.106734+0000","last_became_active":"2023-11-02T15:43:57.118878+0000","last_became_peered":"2023-11-02T15:43:57.118878+0000","last_unstale":"2023-11-02T16:13:48.106734+0000","last_undegraded":"2023-11-02T16:13:48.106734+0000","last_fullsized":"2023-11-02T16:13:48.106734+0000","mapping_epoch":387,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":388,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:48.105686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:48.105686+0000","last_clean_scrub_stamp":"2023-11-02T16:13:48.105686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T17:08:39.054141+0000","scrub_duration":0.043874059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":992,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:34.884540+0000","last_change":"2023-11-02T16:13:34.884540+0000","last_active":"2023-11-02T16:13:34.884540+0000","last_peered":"2023-11-02T16:13:34.884540+0000","last_clean":"2023-11-02T16:13:34.884540+0000","last_became_active":"2023-11-02T15:35:17.355491+0000","last_became_peered":"2023-11-02T15:35:17.355491+0000","last_unstale":"2023-11-02T16:13:34.884540+0000","last_undegraded":"2023-11-02T16:13:34.884540+0000","last_fullsized":"2023-11-02T16:13:34.884540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:34.878451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:34.878451+0000","last_clean_scrub_stamp":"2023-11-02T16:13:34.878451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:17:12.137889+0000","scrub_duration":0.031321511000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1000,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:47.887241+0000","last_change":"2023-11-02T16:13:47.894547+0000","last_active":"2023-11-02T16:13:47.887241+0000","last_peered":"2023-11-02T16:13:47.887241+0000","last_clean":"2023-11-02T16:13:47.887241+0000","last_became_active":"2023-11-02T15:59:17.635098+0000","last_became_peered":"2023-11-02T15:59:17.635098+0000","last_unstale":"2023-11-02T16:13:47.887241+0000","last_undegraded":"2023-11-02T16:13:47.887241+0000","last_fullsized":"2023-11-02T16:13:47.887241+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:47.886446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:47.886446+0000","last_clean_scrub_stamp":"2023-11-02T16:13:47.886446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T03:56:34.008046+0000","scrub_duration":0.054713576999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1001,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:44.910862+0000","last_change":"2023-11-02T16:13:44.918964+0000","last_active":"2023-11-02T16:13:44.910862+0000","last_peered":"2023-11-02T16:13:44.910862+0000","last_clean":"2023-11-02T16:13:44.910862+0000","last_became_active":"2023-11-02T15:59:17.465281+0000","last_became_peered":"2023-11-02T15:59:17.465281+0000","last_unstale":"2023-11-02T16:13:44.910862+0000","last_undegraded":"2023-11-02T16:13:44.910862+0000","last_fullsized":"2023-11-02T16:13:44.910862+0000","mapping_epoch":736,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":737,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:44.910100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:44.910100+0000","last_clean_scrub_stamp":"2023-11-02T16:13:44.910100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:11:37.367165+0000","scrub_duration":0.037239766000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":947,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:46.185571+0000","last_change":"2023-11-02T16:13:46.187387+0000","last_active":"2023-11-02T16:13:46.185571+0000","last_peered":"2023-11-02T16:13:46.185571+0000","last_clean":"2023-11-02T16:13:46.185571+0000","last_became_active":"2023-11-02T16:13:16.800309+0000","last_became_peered":"2023-11-02T16:13:16.800309+0000","last_unstale":"2023-11-02T16:13:46.185571+0000","last_undegraded":"2023-11-02T16:13:46.185571+0000","last_fullsized":"2023-11-02T16:13:46.185571+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:46.184801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:46.184801+0000","last_clean_scrub_stamp":"2023-11-02T16:13:46.184801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:12:41.535289+0000","scrub_duration":0.046431591000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1016,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:46.107294+0000","last_change":"2023-11-02T16:13:46.107294+0000","last_active":"2023-11-02T16:13:46.107294+0000","last_peered":"2023-11-02T16:13:46.107294+0000","last_clean":"2023-11-02T16:13:46.107294+0000","last_became_active":"2023-11-02T16:07:27.393646+0000","last_became_peered":"2023-11-02T16:07:27.393646+0000","last_unstale":"2023-11-02T16:13:46.107294+0000","last_undegraded":"2023-11-02T16:13:46.107294+0000","last_fullsized":"2023-11-02T16:13:46.107294+0000","mapping_epoch":875,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":876,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:46.104828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:46.104828+0000","last_clean_scrub_stamp":"2023-11-02T16:13:46.104828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T19:46:36.744296+0000","scrub_duration":0.045205897000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1015,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:37.991837+0000","last_change":"2023-11-02T16:13:37.997167+0000","last_active":"2023-11-02T16:13:37.991837+0000","last_peered":"2023-11-02T16:13:37.991837+0000","last_clean":"2023-11-02T16:13:37.991837+0000","last_became_active":"2023-11-02T16:13:16.935691+0000","last_became_peered":"2023-11-02T16:13:16.935691+0000","last_unstale":"2023-11-02T16:13:37.991837+0000","last_undegraded":"2023-11-02T16:13:37.991837+0000","last_fullsized":"2023-11-02T16:13:37.991837+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:37.991050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:37.991050+0000","last_clean_scrub_stamp":"2023-11-02T16:13:37.991050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-04T00:03:55.186070+0000","scrub_duration":0.059452926000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1011,"reported_epoch":1030,"state":"active+clean","last_fresh":"2023-11-02T16:13:40.207732+0000","last_change":"2023-11-02T16:13:40.209753+0000","last_active":"2023-11-02T16:13:40.207732+0000","last_peered":"2023-11-02T16:13:40.207732+0000","last_clean":"2023-11-02T16:13:40.207732+0000","last_became_active":"2023-11-02T16:13:16.827572+0000","last_became_peered":"2023-11-02T16:13:16.827572+0000","last_unstale":"2023-11-02T16:13:40.207732+0000","last_undegraded":"2023-11-02T16:13:40.207732+0000","last_fullsized":"2023-11-02T16:13:40.207732+0000","mapping_epoch":1023,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":1024,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-02T16:13:40.206988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-02T16:13:40.206988+0000","last_clean_scrub_stamp":"2023-11-02T16:13:40.206988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-03T21:15:17.710464+0000","scrub_duration":0.037045162,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":1023,"seq":4393751544261,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63856,"kb_used_data":10664,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307984,"statfs":{"total":96636764160,"available":96571375616,"internally_reserved":0,"allocated":10919936,"data_stored":6332119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5616.9430000000002,"5min":1135.9110000000001,"15min":385.58600000000001},"min":{"1min":7.1020000000000003,"5min":4.7069999999999999,"15min":4.665},"max":{"1min":83975.547999999995,"5min":83975.547999999995,"15min":83975.547999999995},"last":9.9160000000000004},{"interface":"front","average":{"1min":5616.5860000000002,"5min":1135.04,"15min":386.13299999999998},"min":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":4.4089999999999998},"max":{"1min":83964.555999999997,"5min":83964.555999999997,"15min":83964.555999999997},"last":7.5979999999999999}]},{"osd":1,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":4683.7309999999998,"5min":953.01900000000001,"15min":2132.357},"min":{"1min":7.867,"5min":6.665,"15min":4.79},"max":{"1min":83982.972999999998,"5min":83982.972999999998,"15min":83982.972999999998},"last":13.94},{"interface":"front","average":{"1min":4686.8530000000001,"5min":952.25599999999997,"15min":2122.1689999999999},"min":{"1min":9.8149999999999995,"5min":7.5780000000000003,"15min":5.2549999999999999},"max":{"1min":83986.676000000007,"5min":83986.676000000007,"15min":83986.676000000007},"last":13.031000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":28065.929,"5min":5631.75,"15min":1886.682},"min":{"1min":16.675999999999998,"5min":8.3160000000000007,"15min":7.4100000000000001},"max":{"1min":84020.403000000006,"5min":84020.403000000006,"15min":84020.403000000006},"last":16.021000000000001},{"interface":"front","average":{"1min":28049.703000000001,"5min":5624.634,"15min":1885.133},"min":{"1min":12.246,"5min":6.431,"15min":6.431},"max":{"1min":83967.169999999998,"5min":83967.169999999998,"15min":83967.169999999998},"last":17.672000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":32909.127999999997,"5min":7672.0479999999998,"15min":2568.2869999999998},"min":{"1min":11.579000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"max":{"1min":83944.991999999998,"5min":83944.991999999998,"15min":83944.991999999998},"last":19.68},{"interface":"front","average":{"1min":31255.992999999999,"5min":7341.4570000000003,"15min":2455.7199999999998},"min":{"1min":9.3640000000000008,"5min":5.9580000000000002,"15min":5.9580000000000002},"max":{"1min":62280.218000000001,"5min":64583.038,"15min":64583.038},"last":11.273}]},{"osd":5,"last update":"Thu Nov 2 16:13:12 2023","interfaces":[{"interface":"back","average":{"1min":5630.2510000000002,"5min":1145.056,"15min":393.048},"min":{"1min":9.6769999999999996,"5min":8.3469999999999995,"15min":7.5819999999999999},"max":{"1min":84121.206000000006,"5min":84121.206000000006,"15min":84121.206000000006},"last":18.247},{"interface":"front","average":{"1min":5616.6610000000001,"5min":1136.4970000000001,"15min":391.23599999999999},"min":{"1min":6.1079999999999997,"5min":6.1079999999999997,"15min":6.1079999999999997},"max":{"1min":83963.747000000003,"5min":83963.747000000003,"15min":83963.747000000003},"last":21.373000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":28040.688999999998,"5min":5623.3860000000004,"15min":3737.663},"min":{"1min":10.163,"5min":7.1299999999999999,"15min":6.508},"max":{"1min":83965.184999999998,"5min":83965.184999999998,"15min":83965.184999999998},"last":14.994},{"interface":"front","average":{"1min":28188.545999999998,"5min":5656.5320000000002,"15min":3750.1060000000002},"min":{"1min":18.253,"5min":10.041,"15min":8.3089999999999993},"max":{"1min":84388.565000000002,"5min":84388.565000000002,"15min":84388.565000000002},"last":20.033000000000001}]},{"osd":7,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":21146.406999999999,"5min":4248.3059999999996,"15min":1438.413},"min":{"1min":15.795,"5min":12.145,"15min":12.145},"max":{"1min":84392.846999999994,"5min":84392.846999999994,"15min":84392.846999999994},"last":16.571999999999999},{"interface":"front","average":{"1min":21039.304,"5min":4226.5299999999997,"15min":1429.376},"min":{"1min":13.507,"5min":10.489000000000001,"15min":10.489000000000001},"max":{"1min":83945.675000000003,"5min":83945.675000000003,"15min":83945.675000000003},"last":22.239000000000001}]}]},{"osd":1,"up_from":16,"seq":68719477197,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104372,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267468,"statfs":{"total":96636764160,"available":96529887232,"internally_reserved":0,"allocated":11317248,"data_stored":6729954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:53 2023","interfaces":[{"interface":"back","average":{"1min":9.9589999999999996,"5min":10.923,"15min":17.305},"min":{"1min":4.7679999999999998,"5min":4.5650000000000004,"15min":4.5650000000000004},"max":{"1min":20.492999999999999,"5min":81.370999999999995,"15min":452.74799999999999},"last":6.0789999999999997},{"interface":"front","average":{"1min":10.323,"5min":11.276,"15min":18.335999999999999},"min":{"1min":6.1079999999999997,"5min":5.2389999999999999,"15min":4.8780000000000001},"max":{"1min":28.196999999999999,"5min":82.305999999999997,"15min":418.30500000000001},"last":11.704000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:10:52 2023","interfaces":[{"interface":"back","average":{"1min":13.831,"5min":17.164999999999999,"15min":23.195},"min":{"1min":7.5620000000000003,"5min":6.5060000000000002,"15min":6.5060000000000002},"max":{"1min":28.725000000000001,"5min":133.536,"15min":419.51400000000001},"last":9.2910000000000004},{"interface":"front","average":{"1min":15.034000000000001,"5min":17.501000000000001,"15min":25.154},"min":{"1min":8.0210000000000008,"5min":5.944,"15min":5.6429999999999998},"max":{"1min":27.068000000000001,"5min":123.39400000000001,"15min":535.63599999999997},"last":5.3380000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":12.576000000000001,"5min":13.050000000000001,"15min":21.103999999999999},"min":{"1min":6.8769999999999998,"5min":5.5720000000000001,"15min":5.5720000000000001},"max":{"1min":19.425000000000001,"5min":64.453000000000003,"15min":549.92200000000003},"last":8.7309999999999999},{"interface":"front","average":{"1min":12.166,"5min":15.949999999999999,"15min":24.456},"min":{"1min":7.3369999999999997,"5min":7.3369999999999997,"15min":7.3369999999999997},"max":{"1min":28.93,"5min":108.244,"15min":542.60500000000002},"last":12.243}]},{"osd":4,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":13.579000000000001,"5min":109.658,"15min":49.033000000000001},"min":{"1min":9.391,"5min":7.8819999999999997,"15min":6.1989999999999998},"max":{"1min":27.463000000000001,"5min":5389.4219999999996,"15min":5389.4219999999996},"last":13.313000000000001},{"interface":"front","average":{"1min":13.228999999999999,"5min":110.251,"15min":50.997999999999998},"min":{"1min":8.8130000000000006,"5min":7.173,"15min":5.8680000000000003},"max":{"1min":20.852,"5min":5473.8580000000002,"15min":5473.8580000000002},"last":9.8569999999999993}]},{"osd":5,"last update":"Thu Nov 2 16:13:53 2023","interfaces":[{"interface":"back","average":{"1min":13.339,"5min":15.272,"15min":24.417999999999999},"min":{"1min":8.609,"5min":7.6790000000000003,"15min":7.2009999999999996},"max":{"1min":36.536999999999999,"5min":71.225999999999999,"15min":529.93499999999995},"last":15.949999999999999},{"interface":"front","average":{"1min":16.213999999999999,"5min":18.603000000000002,"15min":26.672999999999998},"min":{"1min":10.413,"5min":9.5839999999999996,"15min":7.2320000000000002},"max":{"1min":29.494,"5min":122.14,"15min":555.77700000000004},"last":18.329000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":14.141,"5min":20.727,"15min":1552.6310000000001},"min":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1400000000000006},"max":{"1min":21.079000000000001,"5min":245.36500000000001,"15min":73455.679000000004},"last":16.763000000000002},{"interface":"front","average":{"1min":14.130000000000001,"5min":16.395,"15min":1549.3979999999999},"min":{"1min":9.9269999999999996,"5min":7.181,"15min":6.9969999999999999},"max":{"1min":21.965,"5min":124.36499999999999,"15min":73453.190000000002},"last":10.523999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:13:37 2023","interfaces":[{"interface":"back","average":{"1min":13.800000000000001,"5min":16.815000000000001,"15min":20.478999999999999},"min":{"1min":6.8570000000000002,"5min":6.8570000000000002,"15min":6.8570000000000002},"max":{"1min":21.863,"5min":91.019999999999996,"15min":91.019999999999996},"last":12.807},{"interface":"front","average":{"1min":17.292999999999999,"5min":20.710999999999999,"15min":23.553999999999998},"min":{"1min":8.6479999999999997,"5min":8.6479999999999997,"15min":8.6479999999999997},"max":{"1min":25.082999999999998,"5min":133.40000000000001,"15min":133.40000000000001},"last":17.544}]}]},{"osd":0,"up_from":16,"seq":68719477222,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81900,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94289940,"statfs":{"total":96636764160,"available":96552898560,"internally_reserved":0,"allocated":10915840,"data_stored":6331258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 2 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":8.2629999999999999,"5min":10.585000000000001,"15min":77.421999999999997},"min":{"1min":5.6280000000000001,"5min":5.0170000000000003,"15min":4.3399999999999999},"max":{"1min":13.1,"5min":84.331999999999994,"15min":5781.2129999999997},"last":8.5600000000000005},{"interface":"front","average":{"1min":8.9749999999999996,"5min":9.8949999999999996,"15min":74.441000000000003},"min":{"1min":5.5940000000000003,"5min":5.3959999999999999,"15min":5.1749999999999998},"max":{"1min":23.827999999999999,"5min":73.094999999999999,"15min":5443.4889999999996},"last":6.7050000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:07 2023","interfaces":[{"interface":"back","average":{"1min":15.781000000000001,"5min":16.756,"15min":15.430999999999999},"min":{"1min":8.1539999999999999,"5min":6.5170000000000003,"15min":5.0970000000000004},"max":{"1min":101.839,"5min":130.839,"15min":196.92099999999999},"last":10.701000000000001},{"interface":"front","average":{"1min":17.545999999999999,"5min":17.922999999999998,"15min":15.737},"min":{"1min":8.6110000000000007,"5min":5.391,"15min":5.391},"max":{"1min":133.68000000000001,"5min":198.71000000000001,"15min":198.71000000000001},"last":9.4960000000000004}]},{"osd":3,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":16.105,"5min":15.993,"15min":17.158000000000001},"min":{"1min":8.7569999999999997,"5min":7.9219999999999997,"15min":7.46},"max":{"1min":40.524999999999999,"5min":83.388000000000005,"15min":234.29300000000001},"last":13.103999999999999},{"interface":"front","average":{"1min":15.180999999999999,"5min":15.614000000000001,"15min":15.147},"min":{"1min":6.9050000000000002,"5min":6.9050000000000002,"15min":6.6070000000000002},"max":{"1min":62.927,"5min":124.72199999999999,"15min":151.30099999999999},"last":13.798}]},{"osd":4,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":12.595000000000001,"5min":17483.812999999998,"15min":5839.665},"min":{"1min":7.7169999999999996,"5min":7.7169999999999996,"15min":6.3200000000000003},"max":{"1min":21.286000000000001,"5min":66161.077000000005,"15min":66161.077000000005},"last":14.493},{"interface":"front","average":{"1min":16.332000000000001,"5min":17488.550999999999,"15min":5842.2960000000003},"min":{"1min":11.327,"5min":9.9740000000000002,"15min":8.9600000000000009},"max":{"1min":34.182000000000002,"5min":66163.926000000007,"15min":66163.926000000007},"last":15.813000000000001}]},{"osd":5,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":13.404,"5min":17.375,"15min":17.523},"min":{"1min":11.212,"5min":9.1359999999999992,"15min":7.8520000000000003},"max":{"1min":16.760999999999999,"5min":92.465999999999994,"15min":196.56800000000001},"last":15.151999999999999},{"interface":"front","average":{"1min":16.466999999999999,"5min":20.856999999999999,"15min":21.238},"min":{"1min":13.180999999999999,"5min":10.589,"15min":9.1449999999999996},"max":{"1min":19.242999999999999,"5min":93.923000000000002,"15min":197.16999999999999},"last":16.503}]},{"osd":6,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":21.327999999999999,"5min":20.154,"15min":1772.442},"min":{"1min":15.074,"5min":9.6940000000000008,"15min":8.6180000000000003},"max":{"1min":92.451999999999998,"5min":92.451999999999998,"15min":71055.725999999995},"last":17.189},{"interface":"front","average":{"1min":14.882,"5min":15.442,"15min":1770.1980000000001},"min":{"1min":9.7959999999999994,"5min":7.7320000000000002,"15min":7.7320000000000002},"max":{"1min":28.565999999999999,"5min":85.048000000000002,"15min":71055.100000000006},"last":10.036}]},{"osd":7,"last update":"Thu Nov 2 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":19.626000000000001,"5min":22.398,"15min":29.359999999999999},"min":{"1min":13.561999999999999,"5min":10.944000000000001,"15min":10.944000000000001},"max":{"1min":40.588000000000001,"5min":110.238,"15min":205.88999999999999},"last":17.992999999999999},{"interface":"front","average":{"1min":12.699999999999999,"5min":14.935,"15min":22.003},"min":{"1min":8.0090000000000003,"5min":6.9889999999999999,"15min":6.9889999999999999},"max":{"1min":34.991,"5min":84.650999999999996,"15min":155.96600000000001},"last":12.058}]}]},{"osd":6,"up_from":387,"seq":1662152343964,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":8.3919999999999995,"5min":14.112,"15min":20.099},"min":{"1min":4.508,"5min":4.508,"15min":4.508},"max":{"1min":13.715,"5min":208.60499999999999,"15min":572.92100000000005},"last":12.890000000000001},{"interface":"front","average":{"1min":10.018000000000001,"5min":13.913,"15min":21.109000000000002},"min":{"1min":5.3300000000000001,"5min":4.7539999999999996,"15min":4.0419999999999998},"max":{"1min":19.815000000000001,"5min":172.51499999999999,"15min":630.37900000000002},"last":15.173}]},{"osd":1,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":12.505000000000001,"5min":16.201000000000001,"15min":1146.796},"min":{"1min":6.8890000000000002,"5min":5.5990000000000002,"15min":5.04},"max":{"1min":56.890000000000001,"5min":143.55699999999999,"15min":41558.328999999998},"last":6.5789999999999997},{"interface":"front","average":{"1min":11.218999999999999,"5min":16.899999999999999,"15min":1147.8579999999999},"min":{"1min":6.0599999999999996,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":20.495999999999999,"5min":176.839,"15min":41560.286999999997},"last":15.932}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":11.632,"5min":14.904999999999999,"15min":24.402000000000001},"min":{"1min":7.0650000000000004,"5min":6.4859999999999998,"15min":6.4859999999999998},"max":{"1min":16.527999999999999,"5min":103.929,"15min":631.41399999999999},"last":7.3109999999999999},{"interface":"front","average":{"1min":11.667999999999999,"5min":16.355,"15min":24.695},"min":{"1min":7.5010000000000003,"5min":6.5179999999999998,"15min":6.5179999999999998},"max":{"1min":14.561999999999999,"5min":144.18100000000001,"15min":578.42100000000005},"last":8.8510000000000009}]},{"osd":3,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.462,"5min":22.088999999999999,"15min":24.111000000000001},"min":{"1min":8.4540000000000006,"5min":8.4540000000000006,"15min":6.4480000000000004},"max":{"1min":55.115000000000002,"5min":260.11700000000002,"15min":633.899},"last":13.414999999999999},{"interface":"front","average":{"1min":12.683,"5min":18.876000000000001,"15min":24.088999999999999},"min":{"1min":8.0109999999999992,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":17.535,"5min":188.101,"15min":582.32399999999996},"last":9.4269999999999996}]},{"osd":4,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":14.535,"5min":4461.1090000000004,"15min":1505.2059999999999},"min":{"1min":9.8070000000000004,"5min":7.7309999999999999,"15min":6.79},"max":{"1min":21.734000000000002,"5min":67109.214999999997,"15min":67109.214999999997},"last":9.9570000000000007},{"interface":"front","average":{"1min":11.736000000000001,"5min":4422.7439999999997,"15min":1494.021},"min":{"1min":6.992,"5min":6.992,"15min":6.992},"max":{"1min":20.512,"5min":66779.236000000004,"15min":66779.236000000004},"last":16.683}]},{"osd":5,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.187000000000001,"5min":21.879999999999999,"15min":29.353000000000002},"min":{"1min":7.9299999999999997,"5min":7.6609999999999996,"15min":7.6609999999999996},"max":{"1min":54.112000000000002,"5min":281.464,"15min":695.39700000000005},"last":11.503},{"interface":"front","average":{"1min":13.478,"5min":18.571000000000002,"15min":26.082000000000001},"min":{"1min":8.2400000000000002,"5min":6.7290000000000001,"15min":6.7290000000000001},"max":{"1min":22.428999999999998,"5min":222.58799999999999,"15min":577.46299999999997},"last":17.427}]},{"osd":7,"last update":"Thu Nov 2 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":17.585000000000001,"5min":21.940999999999999,"15min":22.190000000000001},"min":{"1min":11.49,"5min":10.291,"15min":10.291},"max":{"1min":50.445999999999998,"5min":246.16200000000001,"15min":246.16200000000001},"last":14.032999999999999},{"interface":"front","average":{"1min":15.438000000000001,"5min":19.43,"15min":20.454000000000001},"min":{"1min":10.507999999999999,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":18.666,"5min":173.136,"15min":173.136},"last":11.785}]}]},{"osd":5,"up_from":14,"seq":60129542614,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64244,"kb_used_data":11052,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307596,"statfs":{"total":96636764160,"available":96570978304,"internally_reserved":0,"allocated":11317248,"data_stored":6729942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3300000000000001,"5min":11.696,"15min":11.385999999999999},"min":{"1min":4.6520000000000001,"5min":4.4059999999999997,"15min":4.2949999999999999},"max":{"1min":20.449000000000002,"5min":109.922,"15min":109.922},"last":9.641},{"interface":"front","average":{"1min":10.396000000000001,"5min":11.989000000000001,"15min":12.337999999999999},"min":{"1min":5.2400000000000002,"5min":4.8040000000000003,"15min":4.6539999999999999},"max":{"1min":28.548999999999999,"5min":66.909999999999997,"15min":170.28899999999999},"last":13.705}]},{"osd":1,"last update":"Thu Nov 2 16:13:53 2023","interfaces":[{"interface":"back","average":{"1min":10.432,"5min":10.702999999999999,"15min":850.26999999999998},"min":{"1min":6.96,"5min":4.8789999999999996,"15min":4.8789999999999996},"max":{"1min":19.187999999999999,"5min":34.581000000000003,"15min":42105.105000000003},"last":11.050000000000001},{"interface":"front","average":{"1min":13.891999999999999,"5min":14.109,"15min":856.20299999999997},"min":{"1min":9.2629999999999999,"5min":7.3319999999999999,"15min":6.508},"max":{"1min":23.559000000000001,"5min":35.615000000000002,"15min":42107.510000000002},"last":14.372}]},{"osd":2,"last update":"Thu Nov 2 16:11:09 2023","interfaces":[{"interface":"back","average":{"1min":11.429,"5min":17.815000000000001,"15min":16.571999999999999},"min":{"1min":7.173,"5min":6.4329999999999998,"15min":6.4329999999999998},"max":{"1min":30.547000000000001,"5min":190.76400000000001,"15min":190.76400000000001},"last":16.568000000000001},{"interface":"front","average":{"1min":14.645,"5min":20.029,"15min":17.998000000000001},"min":{"1min":9.6690000000000005,"5min":7.2220000000000004,"15min":6.7590000000000003},"max":{"1min":28.210999999999999,"5min":139.18700000000001,"15min":166.79300000000001},"last":10.273999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:13:53 2023","interfaces":[{"interface":"back","average":{"1min":12.092000000000001,"5min":14.628,"15min":15.558},"min":{"1min":7.532,"5min":5.8739999999999997,"15min":5.8739999999999997},"max":{"1min":24.792999999999999,"5min":149.886,"15min":172.952},"last":7.9189999999999996},{"interface":"front","average":{"1min":16.509,"5min":19.588000000000001,"15min":18.529},"min":{"1min":11.519,"5min":9.4329999999999998,"15min":6.2839999999999998},"max":{"1min":30.466999999999999,"5min":163.79499999999999,"15min":231.691},"last":12.824}]},{"osd":4,"last update":"Thu Nov 2 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":16.256,"5min":8367.366,"15min":2801.4569999999999},"min":{"1min":8.5039999999999996,"5min":8.5039999999999996,"15min":8.5039999999999996},"max":{"1min":52.700000000000003,"5min":66734.365999999995,"15min":66734.365999999995},"last":17.677},{"interface":"front","average":{"1min":18.460999999999999,"5min":8369.3950000000004,"15min":2801.703},"min":{"1min":6.7409999999999997,"5min":6.7409999999999997,"15min":6.7409999999999997},"max":{"1min":105.985,"5min":66719.994999999995,"15min":66719.994999999995},"last":16.212}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":15.371,"5min":19.039999999999999,"15min":2608.2600000000002},"min":{"1min":8.1110000000000007,"5min":7.8140000000000001,"15min":6.5330000000000004},"max":{"1min":31.577000000000002,"5min":247.161,"15min":74994.611999999994},"last":12.157999999999999},{"interface":"front","average":{"1min":13.83,"5min":18.111000000000001,"15min":2610.2930000000001},"min":{"1min":7.4080000000000004,"5min":7.4080000000000004,"15min":7.4080000000000004},"max":{"1min":21.225000000000001,"5min":241.91,"15min":74995.471000000005},"last":19.510999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":20.102,"5min":19.018000000000001,"15min":23.599},"min":{"1min":10.536,"5min":10.536,"15min":9.0150000000000006},"max":{"1min":106.866,"5min":106.866,"15min":112.215},"last":20.465},{"interface":"front","average":{"1min":15.651999999999999,"5min":17.925999999999998,"15min":25.334},"min":{"1min":10.284000000000001,"5min":9.5399999999999991,"15min":9.5399999999999991},"max":{"1min":28.260000000000002,"5min":93.725999999999999,"15min":103.886},"last":18.391999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542616,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":11.961,"5min":21.25,"15min":14.602},"min":{"1min":5.4219999999999997,"5min":5.0030000000000001,"15min":4.5640000000000001},"max":{"1min":74.665000000000006,"5min":302.62700000000001,"15min":302.62700000000001},"last":9.8859999999999992},{"interface":"front","average":{"1min":16.593,"5min":24.837,"15min":18.509},"min":{"1min":8.8499999999999996,"5min":8.125,"15min":5.9379999999999997},"max":{"1min":91.230999999999995,"5min":289.18200000000002,"15min":289.18200000000002},"last":13.518000000000001}]},{"osd":1,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":8.1400000000000006,"5min":26.207999999999998,"15min":894.40300000000002},"min":{"1min":5.5780000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"max":{"1min":10.657999999999999,"5min":682.15599999999995,"15min":38151.266000000003},"last":6.4960000000000004},{"interface":"front","average":{"1min":11.291,"5min":32.579999999999998,"15min":900.13999999999999},"min":{"1min":7.1660000000000004,"5min":7.1660000000000004,"15min":7.1660000000000004},"max":{"1min":16.356000000000002,"5min":685.01300000000003,"15min":38172.237000000001},"last":17.163}]},{"osd":2,"last update":"Thu Nov 2 16:11:40 2023","interfaces":[{"interface":"back","average":{"1min":69.158000000000001,"5min":28.806999999999999,"15min":19.213999999999999},"min":{"1min":7.7859999999999996,"5min":6.5800000000000001,"15min":6.4699999999999998},"max":{"1min":606.88599999999997,"5min":606.88599999999997,"15min":606.88599999999997},"last":9.1120000000000001},{"interface":"front","average":{"1min":73.319000000000003,"5min":31.344000000000001,"15min":19.311},"min":{"1min":9.1910000000000007,"5min":8.2539999999999996,"15min":6.0110000000000001},"max":{"1min":610.29899999999998,"5min":610.29899999999998,"15min":610.29899999999998},"last":12.909000000000001}]},{"osd":3,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":14.381,"5min":27.777999999999999,"15min":18.989999999999998},"min":{"1min":7.4180000000000001,"5min":6.048,"15min":6.048},"max":{"1min":46.685000000000002,"5min":683.65200000000004,"15min":683.65200000000004},"last":11.099},{"interface":"front","average":{"1min":14.340999999999999,"5min":26.373000000000001,"15min":18.59},"min":{"1min":6.4000000000000004,"5min":6.4000000000000004,"15min":6.4000000000000004},"max":{"1min":65.313000000000002,"5min":609.31899999999996,"15min":609.31899999999996},"last":14.266999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":22.913,"5min":41.131,"15min":24.827999999999999},"min":{"1min":8.8740000000000006,"5min":8.4670000000000005,"15min":6.8540000000000001},"max":{"1min":192.56700000000001,"5min":607.48199999999997,"15min":607.48199999999997},"last":10.478},{"interface":"front","average":{"1min":21.675000000000001,"5min":28.567,"15min":19.670999999999999},"min":{"1min":7.1609999999999996,"5min":6.3659999999999997,"15min":6.3659999999999997},"max":{"1min":191.435,"5min":287.95100000000002,"15min":287.95100000000002},"last":16.167000000000002}]},{"osd":6,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":13.749000000000001,"5min":25.651,"15min":2144.5120000000002},"min":{"1min":6.8760000000000003,"5min":6.8760000000000003,"15min":6.8760000000000003},"max":{"1min":24.265999999999998,"5min":688.24300000000005,"15min":75975.607000000004},"last":15.529},{"interface":"front","average":{"1min":14.723000000000001,"5min":27.239999999999998,"15min":2143.9769999999999},"min":{"1min":9.7739999999999991,"5min":7.8680000000000003,"15min":6.702},"max":{"1min":19.850999999999999,"5min":610.89800000000002,"15min":75971.296000000002},"last":12.321}]},{"osd":7,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":16.059000000000001,"5min":139.43199999999999,"15min":66.170000000000002},"min":{"1min":13.545,"5min":10.411,"15min":10.411},"max":{"1min":18.268000000000001,"5min":611.59799999999996,"15min":611.59799999999996},"last":9.5969999999999995},{"interface":"front","average":{"1min":13.574999999999999,"5min":139.542,"15min":57.654000000000003},"min":{"1min":8.984,"5min":8.609,"15min":7.0410000000000004},"max":{"1min":17.788,"5min":629.07500000000005,"15min":629.07500000000005},"last":17.896000000000001}]}]},{"osd":3,"up_from":736,"seq":3161095930316,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103980,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94267860,"statfs":{"total":96636764160,"available":96530288640,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":14.256,"5min":12.529999999999999,"15min":14.252000000000001},"min":{"1min":4.673,"5min":4.3550000000000004,"15min":4.3550000000000004},"max":{"1min":79.668000000000006,"5min":101.90600000000001,"15min":159.828},"last":6.7220000000000004},{"interface":"front","average":{"1min":18.561,"5min":13.141999999999999,"15min":14.512},"min":{"1min":5.7389999999999999,"5min":5.2060000000000004,"15min":4.4000000000000004},"max":{"1min":131.036,"5min":131.036,"15min":153.459},"last":7.6879999999999997}]},{"osd":1,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":19.010000000000002,"5min":13.263,"15min":386.18900000000002},"min":{"1min":7.3339999999999996,"5min":5.2999999999999998,"15min":5.2999999999999998},"max":{"1min":152.60900000000001,"5min":152.60900000000001,"15min":20852.026000000002},"last":7.0149999999999997},{"interface":"front","average":{"1min":20.308,"5min":15.276999999999999,"15min":382.14400000000001},"min":{"1min":8.9550000000000001,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":93.549999999999997,"5min":93.549999999999997,"15min":20733.266},"last":11.782}]},{"osd":2,"last update":"Thu Nov 2 16:11:38 2023","interfaces":[{"interface":"back","average":{"1min":10.996,"5min":15.276999999999999,"15min":17.300000000000001},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":15.451000000000001,"5min":82.420000000000002,"15min":180.51499999999999},"last":10.603},{"interface":"front","average":{"1min":12.116,"5min":17.431000000000001,"15min":19.491},"min":{"1min":8.5690000000000008,"5min":6.6219999999999999,"15min":6.6219999999999999},"max":{"1min":14.868,"5min":123.051,"15min":193.43600000000001},"last":9.7590000000000003}]},{"osd":4,"last update":"Thu Nov 2 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":25.468,"5min":22.239999999999998,"15min":21.866},"min":{"1min":8.5199999999999996,"5min":8.0069999999999997,"15min":7.0129999999999999},"max":{"1min":202.30099999999999,"5min":202.30099999999999,"15min":202.30099999999999},"last":13.775},{"interface":"front","average":{"1min":25.638999999999999,"5min":21.823,"15min":22.591000000000001},"min":{"1min":7.375,"5min":7.375,"15min":6.944},"max":{"1min":179.37299999999999,"5min":179.37299999999999,"15min":210.239},"last":17.57}]},{"osd":5,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":32.866999999999997,"5min":22.800000000000001,"15min":23.951000000000001},"min":{"1min":10.111000000000001,"5min":9.0129999999999999,"15min":8.9130000000000003},"max":{"1min":180.82499999999999,"5min":180.82499999999999,"15min":290.07100000000003},"last":13.250999999999999},{"interface":"front","average":{"1min":28.981000000000002,"5min":19.195,"15min":20.167999999999999},"min":{"1min":8.7330000000000005,"5min":8.4949999999999992,"15min":6.8220000000000001},"max":{"1min":170.89599999999999,"5min":170.89599999999999,"15min":268.577},"last":16.675000000000001}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":25.896999999999998,"5min":19.382999999999999,"15min":1643.183},"min":{"1min":12.167,"5min":7.1180000000000003,"15min":7.1180000000000003},"max":{"1min":102.196,"5min":102.196,"15min":76086.231},"last":15.1},{"interface":"front","average":{"1min":20.253,"5min":19.183,"15min":1650.2560000000001},"min":{"1min":10.005000000000001,"5min":9.4280000000000008,"15min":9.4280000000000008},"max":{"1min":62.909999999999997,"5min":71.622,"15min":76087.114000000001},"last":12.327999999999999}]},{"osd":7,"last update":"Thu Nov 2 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":30.446000000000002,"5min":20.318999999999999,"15min":23.869},"min":{"1min":12.579000000000001,"5min":10.125,"15min":8.8870000000000005},"max":{"1min":178.74299999999999,"5min":178.74299999999999,"15min":178.74299999999999},"last":18.332999999999998},{"interface":"front","average":{"1min":24.611999999999998,"5min":19.600999999999999,"15min":24.111000000000001},"min":{"1min":10.590999999999999,"5min":9.5419999999999998,"15min":9.5419999999999998},"max":{"1min":118.634,"5min":118.634,"15min":140.87299999999999},"last":15.361000000000001}]}]},{"osd":7,"up_from":875,"seq":3758096384439,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63852,"kb_used_data":10660,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94307988,"statfs":{"total":96636764160,"available":96571379712,"internally_reserved":0,"allocated":10915840,"data_stored":6332114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":12.968999999999999,"5min":12.625999999999999,"15min":15.327999999999999},"min":{"1min":5.4749999999999996,"5min":4.7430000000000003,"15min":4.7430000000000003},"max":{"1min":87.353999999999999,"5min":87.353999999999999,"15min":87.353999999999999},"last":5.3099999999999996},{"interface":"front","average":{"1min":12.054,"5min":11.366,"15min":17.962},"min":{"1min":5.3380000000000001,"5min":4.8520000000000003,"15min":4.8520000000000003},"max":{"1min":62.587000000000003,"5min":62.587000000000003,"15min":97.984999999999999},"last":8.7270000000000003}]},{"osd":1,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":14.287000000000001,"5min":14.279,"15min":21.698},"min":{"1min":6.2439999999999998,"5min":5.8220000000000001,"15min":5.8220000000000001},"max":{"1min":34.357999999999997,"5min":48.886000000000003,"15min":178.56},"last":6.9429999999999996},{"interface":"front","average":{"1min":10.861000000000001,"5min":11.736000000000001,"15min":14.888999999999999},"min":{"1min":6.6029999999999998,"5min":4.9189999999999996,"15min":4.9189999999999996},"max":{"1min":32.017000000000003,"5min":47.533999999999999,"15min":66.081999999999994},"last":11.701000000000001}]},{"osd":2,"last update":"Thu Nov 2 16:11:32 2023","interfaces":[{"interface":"back","average":{"1min":13.603999999999999,"5min":18.32,"15min":19.437999999999999},"min":{"1min":7.8700000000000001,"5min":6.5010000000000003,"15min":6.5010000000000003},"max":{"1min":37.152000000000001,"5min":116.27200000000001,"15min":116.27200000000001},"last":14.609999999999999},{"interface":"front","average":{"1min":13.606,"5min":19.184999999999999,"15min":20.57},"min":{"1min":7.6680000000000001,"5min":6.7110000000000003,"15min":6.7110000000000003},"max":{"1min":31.850999999999999,"5min":119.336,"15min":119.336},"last":12.798999999999999}]},{"osd":3,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":21.795000000000002,"5min":21.023,"15min":19.797000000000001},"min":{"1min":7.6689999999999996,"5min":7.5579999999999998,"15min":7.5579999999999998},"max":{"1min":143.935,"5min":253.71000000000001,"15min":253.71000000000001},"last":12.177},{"interface":"front","average":{"1min":19.591999999999999,"5min":16.908000000000001,"15min":18.48},"min":{"1min":8.8000000000000007,"5min":7.0389999999999997,"15min":7.0389999999999997},"max":{"1min":76.149000000000001,"5min":101.316,"15min":101.316},"last":16.283000000000001}]},{"osd":4,"last update":"Thu Nov 2 16:13:45 2023","interfaces":[{"interface":"back","average":{"1min":14.881,"5min":9823.1419999999998,"15min":3300.5569999999998},"min":{"1min":7.5549999999999997,"5min":6.7549999999999999,"15min":6.7549999999999999},"max":{"1min":29.866,"5min":66277.987999999998,"15min":66277.987999999998},"last":19.484999999999999},{"interface":"front","average":{"1min":18.414000000000001,"5min":9829.7340000000004,"15min":3300.77},"min":{"1min":9.0220000000000002,"5min":8.6539999999999999,"15min":8.6539999999999999},"max":{"1min":77.519999999999996,"5min":66301.380000000005,"15min":66301.380000000005},"last":17.609999999999999}]},{"osd":5,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":18.657,"5min":20.384,"15min":22.456},"min":{"1min":8.6859999999999999,"5min":8.2059999999999995,"15min":8.2059999999999995},"max":{"1min":86.191999999999993,"5min":263.65699999999998,"15min":263.65699999999998},"last":18.324999999999999},{"interface":"front","average":{"1min":26.029,"5min":22.896999999999998,"15min":24.760000000000002},"min":{"1min":8.8989999999999991,"5min":8.6319999999999997,"15min":8.6319999999999997},"max":{"1min":242.703,"5min":242.703,"15min":242.703},"last":20.298999999999999}]},{"osd":6,"last update":"Thu Nov 2 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":19.016999999999999,"5min":23.106000000000002,"15min":25.989999999999998},"min":{"1min":10.065,"5min":8.4860000000000007,"15min":8.4860000000000007},"max":{"1min":75.099999999999994,"5min":285.56999999999999,"15min":285.56999999999999},"last":15.634},{"interface":"front","average":{"1min":18.521999999999998,"5min":21.626999999999999,"15min":23.535},"min":{"1min":13.172000000000001,"5min":8.2400000000000002,"15min":8.2400000000000002},"max":{"1min":39.274000000000001,"5min":175.82300000000001,"15min":175.82300000000001},"last":16.939}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-02T16:14:15.745 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-02T16:14:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 10 monclient: get_monmap_and_config 2023-11-02T16:14:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 10 monclient: build_initial_monmap 2023-11-02T16:14:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 10 monclient: monmap: 2023-11-02T16:14:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:14:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T16:14:15.845017+0000 2023-11-02T16:14:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T16:14:15.845017+0000 2023-11-02T16:14:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:14:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:0: v1:172.21.15.174:6789/0 mon.noname-a-legacy 2023-11-02T16:14:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:1: v1:172.21.15.175:6789/0 mon.noname-c-legacy 2023-11-02T16:14:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-b 2023-11-02T16:14:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:3: v2:172.21.15.174:3300/0 mon.noname-a 2023-11-02T16:14:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:4: v2:172.21.15.175:3300/0 mon.noname-c 2023-11-02T16:14:15.848 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 1 Processor -- start 2023-11-02T16:14:15.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 1 -- start start 2023-11-02T16:14:15.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.844+0000 7fb61fdf5640 10 monclient: init 2023-11-02T16:14:15.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient: _reopen_session rank -1 2023-11-02T16:14:15.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient: _add_conns ranks=[3,4,0,1,2] 2023-11-02T16:14:15.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb6180b4570 0x7fb618144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-a con 0x7fb6180b4570 addr v2:172.21.15.174:3300/0 2023-11-02T16:14:15.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6180b0ed0 0x7fb6180a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-c con 0x7fb6180b0ed0 addr v2:172.21.15.175:3300/0 2023-11-02T16:14:15.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6180b1cb0 addr v1:172.21.15.174:6789/0 2023-11-02T16:14:15.855 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6180aa1d0 con 0x7fb6180b1cb0 2023-11-02T16:14:15.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6180b4950 con 0x7fb6180b4570 2023-11-02T16:14:15.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6180aa4b0 con 0x7fb6180b0ed0 2023-11-02T16:14:15.856 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): _renew_subs 2023-11-02T16:14:15.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.845+0000 7fb61fdf5640 10 monclient(hunting): authenticate will time out at 3783.187159s 2023-11-02T16:14:15.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6180b0ed0 0x7fb6180a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61db91640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb6180b4570 0x7fb618144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61e392640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb6180b1cb0 0x7fb6180b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:50098/0 (socket says 172.21.15.174:50098) 2023-11-02T16:14:15.857 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61e392640 1 -- 172.21.15.174:0/1030960852 learned_addr learned my addr 172.21.15.174:0/1030960852 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-02T16:14:15.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb6180b0ed0 0x7fb6180a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.174:33920/0 (socket says 172.21.15.174:33920) 2023-11-02T16:14:15.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): get_auth_request con 0x7fb6180b0ed0 auth_method 0 2023-11-02T16:14:15.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.858 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61db91640 10 monclient(hunting): get_auth_request con 0x7fb6180b4570 auth_method 0 2023-11-02T16:14:15.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61db91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61db91640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61db91640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.859 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.846+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.847+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.847+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.860 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.847+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 10 monclient(hunting): handle_auth_done global_id 21613 payload 274 2023-11-02T16:14:15.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 1 -- 172.21.15.174:0/1030960852 >> v2:172.21.15.174:3300/0 conn(0x7fb6180b4570 msgr2=0x7fb618144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-02T16:14:15.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 1 --2- 172.21.15.174:0/1030960852 >> v2:172.21.15.174:3300/0 conn(0x7fb6180b4570 0x7fb618144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 1 -- 172.21.15.174:0/1030960852 >> v1:172.21.15.174:6789/0 conn(0x7fb6180b1cb0 legacy=0x7fb6180b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.861 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 10 monclient: _finish_hunting 0 2023-11-02T16:14:15.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 1 monclient: found mon.noname-c 2023-11-02T16:14:15.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:14:15.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.175:3300/0 2023-11-02T16:14:15.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.848+0000 7fb61d390640 1 -- 172.21.15.174:0/1030960852 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6180a8b50 con 0x7fb6180b0ed0 2023-11-02T16:14:15.862 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.850+0000 7fb61d390640 1 --2- 172.21.15.174:0/1030960852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b0ed0 0x7fb6180a9070 secure :-1 s=READY pgs=5288 cs=0 l=1 rev1=1 crypto rx=0x7fb608009790 tx=0x7fb60802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19143f231c29c624 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:15.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 1 -- 172.21.15.174:0/1030960852 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802f050 con 0x7fb6180b0ed0 2023-11-02T16:14:15.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-11-02T16:14:15.868 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: dump: 2023-11-02T16:14:15.869 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.869 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.869 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.869 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.869 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.870 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.870 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.870 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.870 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.870 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: _finish_auth 0 2023-11-02T16:14:15.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: _check_auth_tickets 2023-11-02T16:14:15.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:14:15.874 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 1 -- 172.21.15.174:0/1030960852 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608031700 con 0x7fb6180b0ed0 2023-11-02T16:14:15.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 5 monclient: authenticate success, global_id 21613 2023-11-02T16:14:15.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 20 monclient: get_monmap_and_config waiting for monmap|config 2023-11-02T16:14:15.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:14:15.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 10 monclient: get_monmap_and_config success 2023-11-02T16:14:15.875 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 1 -- 172.21.15.174:0/1030960852 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608031a90 con 0x7fb6180b0ed0 2023-11-02T16:14:15.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 4 set_mon_vals no callback set 2023-11-02T16:14:15.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:14:15.876 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61cb8f640 10 monclient: dump: 2023-11-02T16:14:15.876 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.877 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.878 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.878 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.878 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.878 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 10 monclient: shutdown 2023-11-02T16:14:15.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:14:15.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1030960852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b0ed0 msgr2=0x7fb6180a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1030960852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b0ed0 0x7fb6180a9070 secure :-1 s=READY pgs=5288 cs=0 l=1 rev1=1 crypto rx=0x7fb608009790 tx=0x7fb60802b7a0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1030960852 shutdown_connections 2023-11-02T16:14:15.879 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1030960852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b0ed0 0x7fb6180a9070 unknown :-1 s=CLOSED pgs=5288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1030960852 >> v2:172.21.15.174:3300/0 conn(0x7fb6180b4570 0x7fb618144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1030960852 >> 172.21.15.174:0/1030960852 conn(0x7fb618065d50 msgr2=0x7fb618066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1030960852 shutdown_connections 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.854+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1030960852 wait complete. 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 10 monclient: build_initial_monmap 2023-11-02T16:14:15.880 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 10 monclient: monmap: 2023-11-02T16:14:15.881 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 0 2023-11-02T16:14:15.881 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-11-02T16:14:15.881 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 0.000000 2023-11-02T16:14:15.881 INFO:teuthology.orchestra.run.smithi174.stderr:created 0.000000 2023-11-02T16:14:15.881 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 0 (unknown) 2023-11-02T16:14:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.noname-a 2023-11-02T16:14:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.noname-b 2023-11-02T16:14:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.noname-c 2023-11-02T16:14:15.882 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 librados: starting msgr at 2023-11-02T16:14:15.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 librados: starting objecter 2023-11-02T16:14:15.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 Processor -- start 2023-11-02T16:14:15.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 -- start start 2023-11-02T16:14:15.883 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 librados: setting wanted keys 2023-11-02T16:14:15.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 1 librados: calling monclient init 2023-11-02T16:14:15.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.855+0000 7fb61fdf5640 10 monclient: init 2023-11-02T16:14:15.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient: _reopen_session rank -1 2023-11-02T16:14:15.884 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient: _add_conns ranks=[0,1,2] 2023-11-02T16:14:15.885 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.885 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-a con 0x7fb6180b0ed0 addr [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-02T16:14:15.885 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 0x7fb6180ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.885 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-b con 0x7fb6180b1cb0 addr [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2023-11-02T16:14:15.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6180b4570 0x7fb6181da170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:15.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): picked mon.noname-c con 0x7fb6180b4570 addr [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2023-11-02T16:14:15.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6180ab630 con 0x7fb6180b0ed0 2023-11-02T16:14:15.886 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6181d9c50 con 0x7fb6180b1cb0 2023-11-02T16:14:15.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): start opening mon connection 2023-11-02T16:14:15.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6181423e0 con 0x7fb6180b4570 2023-11-02T16:14:15.887 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): _renew_subs 2023-11-02T16:14:15.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61fdf5640 10 monclient(hunting): authenticate will time out at 3783.198059s 2023-11-02T16:14:15.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61db91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61d390640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 0x7fb6180ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.856+0000 7fb61e392640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6180b4570 0x7fb6181da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:15.888 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39136/0 (socket says 172.21.15.174:39136) 2023-11-02T16:14:15.889 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 1 -- 172.21.15.174:0/1162636763 learned_addr learned my addr 172.21.15.174:0/1162636763 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-02T16:14:15.889 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): get_auth_request con 0x7fb6180b0ed0 auth_method 0 2023-11-02T16:14:15.889 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.889 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.889 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.890 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): get_auth_request con 0x7fb6180b1cb0 auth_method 0 2023-11-02T16:14:15.890 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.890 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.890 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.890 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): get_auth_request con 0x7fb6180b4570 auth_method 0 2023-11-02T16:14:15.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-11-02T16:14:15.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): _init_auth method 2 2023-11-02T16:14:15.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): _init_auth creating new auth 2023-11-02T16:14:15.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.891 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.892 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61db91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.892 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.892 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.892 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61d390640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.892 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): handle_auth_reply_more payload 9 2023-11-02T16:14:15.893 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-11-02T16:14:15.893 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.857+0000 7fb61e392640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-11-02T16:14:15.893 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 10 monclient(hunting): handle_auth_done global_id 21616 payload 824 2023-11-02T16:14:15.893 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 -- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6180b4570 msgr2=0x7fb6181da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.893 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6180b4570 0x7fb6181da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 -- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 msgr2=0x7fb6180abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:15.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:15.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 10 monclient: _finish_hunting 0 2023-11-02T16:14:15.894 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 monclient: found mon.noname-b 2023-11-02T16:14:15.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-11-02T16:14:15.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.175:3300/0 2023-11-02T16:14:15.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61d390640 1 -- 172.21.15.174:0/1162636763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080093a0 con 0x7fb6180b1cb0 2023-11-02T16:14:15.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.860+0000 7fb61db91640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-02T16:14:15.895 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.862+0000 7fb61d390640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 0x7fb6180ab0f0 secure :-1 s=READY pgs=5289 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb6080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:15.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.862+0000 7fb60e7fc640 10 client.?.objecter ms_handle_connect 0x7fb6180b1cb0 2023-11-02T16:14:15.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.862+0000 7fb60e7fc640 10 client.?.objecter resend_mon_ops 2023-11-02T16:14:15.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608005ba0 con 0x7fb6180b1cb0 2023-11-02T16:14:15.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.896 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: dump: 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.897 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.898 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.898 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.898 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.898 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.898 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: _finish_auth 0 2023-11-02T16:14:15.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: _check_auth_tickets 2023-11-02T16:14:15.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2023-11-02T16:14:15.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608005d00 con 0x7fb6180b1cb0 2023-11-02T16:14:15.899 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: handle_config config(0 keys) v1 2023-11-02T16:14:15.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608033a60 con 0x7fb6180b1cb0 2023-11-02T16:14:15.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 5 monclient: authenticate success, global_id 21616 2023-11-02T16:14:15.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 10 monclient: _renew_subs 2023-11-02T16:14:15.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:14:15.900 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb618140790 con 0x7fb6180b1cb0 2023-11-02T16:14:15.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60ffff640 4 set_mon_vals no callback set 2023-11-02T16:14:15.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2023-11-02T16:14:15.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-11-02T16:14:15.901 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb60e7fc640 10 monclient: dump: 2023-11-02T16:14:15.901 INFO:teuthology.orchestra.run.smithi174.stderr:epoch 1 2023-11-02T16:14:15.902 INFO:teuthology.orchestra.run.smithi174.stderr:fsid 71bb545f-71ad-46c8-9016-794bb3baf3fa 2023-11-02T16:14:15.902 INFO:teuthology.orchestra.run.smithi174.stderr:last_changed 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.902 INFO:teuthology.orchestra.run.smithi174.stderr:created 2023-11-02T15:32:14.751315+0000 2023-11-02T16:14:15.902 INFO:teuthology.orchestra.run.smithi174.stderr:min_mon_release 18 (reef) 2023-11-02T16:14:15.902 INFO:teuthology.orchestra.run.smithi174.stderr:election_strategy: 1 2023-11-02T16:14:15.903 INFO:teuthology.orchestra.run.smithi174.stderr:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2023-11-02T16:14:15.903 INFO:teuthology.orchestra.run.smithi174.stderr:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-11-02T16:14:15.903 INFO:teuthology.orchestra.run.smithi174.stderr:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-02T16:14:15.903 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-02T16:14:15.903 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 10 client.21616.objecter _maybe_request_map subscribing (onetime) to next osd map 2023-11-02T16:14:15.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 10 monclient: _renew_subs 2023-11-02T16:14:15.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:14:15.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181da890 con 0x7fb6180b1cb0 2023-11-02T16:14:15.904 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.866+0000 7fb61fdf5640 1 librados: init done 2023-11-02T16:14:15.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.867+0000 7fb61fdf5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:15.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.867+0000 7fb61fdf5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:15.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.867+0000 7fb61fdf5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:14:15.905 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:15.867+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6180ad590 con 0x7fb6180b1cb0 2023-11-02T16:14:16.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb60803a040 con 0x7fb6180b1cb0 2023-11-02T16:14:16.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb60e7fc640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 0x7fb5fc06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-02T16:14:16.108 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb61db91640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 0x7fb5fc06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-02T16:14:16.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb61db91640 10 monclient: get_auth_request con 0x7fb5fc06bdc0 auth_method 0 2023-11-02T16:14:16.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb60e7fc640 10 client.21616.objecter ms_handle_connect 0x7fb5fc06bdc0 2023-11-02T16:14:16.109 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.106+0000 7fb61db91640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 0x7fb5fc06e280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6140112c0 tx=0x7fb614009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-02T16:14:16.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.108+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1030..1030 src has 247..1030) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb608079c60 con 0x7fb6180b1cb0 2023-11-02T16:14:16.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.108+0000 7fb60e7fc640 10 client.21616.objecter ms_dispatch 0x7fb618000b90 osd_map(1030..1030 src has 247..1030) v4 2023-11-02T16:14:16.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.108+0000 7fb60e7fc640 3 client.21616.objecter handle_osd_map got epochs [1030,1030] > 0 2023-11-02T16:14:16.110 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.108+0000 7fb60e7fc640 3 client.21616.objecter handle_osd_map decoding full epoch 1030 2023-11-02T16:14:16.111 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.108+0000 7fb60e7fc640 20 client.21616.objecter dump_active .. 0 homeless 2023-11-02T16:14:16.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.119+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb608075fa0 con 0x7fb6180b1cb0 2023-11-02T16:14:16.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.119+0000 7fb60e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-11-02T16:14:16.121 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.119+0000 7fb60e7fc640 10 monclient: _finish_command 1 = system:0 2023-11-02T16:14:16.290 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.289+0000 7fb61fdf5640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-11-02T16:14:16.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.289+0000 7fb61fdf5640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-11-02T16:14:16.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.289+0000 7fb61fdf5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.175:3300/0 2023-11-02T16:14:16.291 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.289+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fb6180b3ad0 con 0x7fb6180b1cb0 2023-11-02T16:14:16.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb608075fa0 con 0x7fb6180b1cb0 2023-11-02T16:14:16.498 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60e7fc640 10 monclient: handle_config config(1 keys) v1 2023-11-02T16:14:16.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60ffff640 4 set_mon_vals no callback set 2023-11-02T16:14:16.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60ffff640 10 set_mon_vals mon_health_to_clog = false 2023-11-02T16:14:16.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60e7fc640 1 -- 172.21.15.174:0/1162636763 <== mon.1 v2:172.21.15.175:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb608038020 con 0x7fb6180b1cb0 2023-11-02T16:14:16.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2023-11-02T16:14:16.499 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.496+0000 7fb60e7fc640 10 monclient: _finish_command 2 = system:0 2023-11-02T16:14:16.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 10 librados: watch_flush enter 2023-11-02T16:14:16.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 10 librados: watch_flush exit 2023-11-02T16:14:16.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 20 client.21616.objecter shutdown clearing up homeless session... 2023-11-02T16:14:16.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 10 client.21616.objecter successfully canceled tick 2023-11-02T16:14:16.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 msgr2=0x7fb5fc06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:16.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 0x7fb5fc06e280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6140112c0 tx=0x7fb614009190 comp rx=0 tx=0).stop 2023-11-02T16:14:16.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 10 monclient: shutdown 2023-11-02T16:14:16.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 20 monclient: shutdown discarding 0 pending message(s) 2023-11-02T16:14:16.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 msgr2=0x7fb6180ab0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-02T16:14:16.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 0x7fb6180ab0f0 secure :-1 s=READY pgs=5289 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb6080046c0 comp rx=0 tx=0).stop 2023-11-02T16:14:16.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 shutdown_connections 2023-11-02T16:14:16.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:6800/85844,v1:172.21.15.174:6801/85844] conn(0x7fb5fc06bdc0 0x7fb5fc06e280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:16.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6180b4570 0x7fb6181da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:16.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6180b1cb0 0x7fb6180ab0f0 unknown :-1 s=CLOSED pgs=5289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:16.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 --2- 172.21.15.174:0/1162636763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6180b0ed0 0x7fb6180abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-02T16:14:16.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 >> 172.21.15.174:0/1162636763 conn(0x7fb618065d50 msgr2=0x7fb6180b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-02T16:14:16.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 shutdown_connections 2023-11-02T16:14:16.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 -- 172.21.15.174:0/1162636763 wait complete. 2023-11-02T16:14:16.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-02T16:14:16.498+0000 7fb61fdf5640 1 librados: shutdown 2023-11-02T16:14:16.511 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-02T16:14:16.511 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-02T16:14:16.512 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-02T16:14:16.512 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:14:16.524 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:14:16.514+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86117) UID: 0 2023-11-02T16:14:16.525 INFO:tasks.ceph.osd.0.smithi174.stderr:2023-11-02T16:14:16.516+0000 b47e640 -1 osd.0 1030 *** Got signal Terminated *** 2023-11-02T16:14:19.567 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:31.720 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.572 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:31.725 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.585 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:31.738 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.614 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:31.767 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.617 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:31.713 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.618 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:31.715 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.619 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:31.773 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.633 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:31.730 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:19.634 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:31.788 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:22.615 INFO:tasks.ceph.osd.0:Stopped 2023-11-02T16:14:22.616 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-02T16:14:22.616 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:14:22.628 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:14:22.618+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86126) UID: 0 2023-11-02T16:14:22.629 INFO:tasks.ceph.osd.1.smithi174.stderr:2023-11-02T16:14:22.619+0000 b47e640 -1 osd.1 1032 *** Got signal Terminated *** 2023-11-02T16:14:28.718 INFO:tasks.ceph.osd.1:Stopped 2023-11-02T16:14:28.719 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-02T16:14:28.720 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:14:28.732 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T16:14:28.721+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86130) UID: 0 2023-11-02T16:14:28.733 INFO:tasks.ceph.osd.2.smithi174.stderr:2023-11-02T16:14:28.722+0000 b47e640 -1 osd.2 1035 *** Got signal Terminated *** 2023-11-02T16:14:29.596 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:41.749 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.598 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:41.752 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.617 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:41.770 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.645 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:41.741 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.645 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:41.798 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.647 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:41.744 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.647 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:41.801 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.662 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:41.758 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:29.663 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:41.817 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.745 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:46.899 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.751 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:46.904 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.767 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:41:46.921 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.799 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:46.895 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.800 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:46.896 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.813 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:41:46.909 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.813 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:46.967 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.815 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:46.968 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:34.830 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:41:46.984 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.695 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:01.847 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.696 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:01.850 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.718 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:01.869 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.743 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:01.839 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.747 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:01.844 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.751 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:01.904 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.752 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:01.905 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.762 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:01.858 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:49.771 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:01.924 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.706 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:06.860 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.709 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:06.862 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.718 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:06.871 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.754 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:06.907 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.756 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:06.853 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.757 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:06.910 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.758 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:06.854 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.768 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:06.865 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:14:54.770 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:06.923 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:02.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-02T16:15:05.329 INFO:tasks.ceph.osd.2:Stopped 2023-11-02T16:15:05.330 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-02T16:15:05.331 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:15:05.343 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:15:05.333+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86138) UID: 0 2023-11-02T16:15:05.343 INFO:tasks.ceph.osd.3.smithi174.stderr:2023-11-02T16:15:05.334+0000 b47e640 -1 osd.3 1038 *** Got signal Terminated *** 2023-11-02T16:15:08.287 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-02T16:15:08.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-02T16:15:09.780 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:21.933 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.785 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:21.938 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.802 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:21.955 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.833 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:21.986 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.837 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:21.932 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.837 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:21.934 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.837 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:21.991 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.854 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:21.951 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:09.855 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:22.008 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:11.433 INFO:tasks.ceph.osd.3:Stopped 2023-11-02T16:15:11.434 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-02T16:15:11.434 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:15:11.450 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T16:15:11.436+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86405) UID: 0 2023-11-02T16:15:11.450 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-11-02T16:15:11.437+0000 b47e640 -1 osd.4 1040 *** Got signal Terminated *** 2023-11-02T16:15:14.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-11-02T16:15:14.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-02T16:15:14.798 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:26.951 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.803 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:26.956 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.816 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:26.969 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.852 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:27.005 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.853 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:26.949 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.854 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:27.007 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.855 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:26.951 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.867 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:27.020 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:14.867 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:26.963 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:17.535 INFO:tasks.ceph.osd.4:Stopped 2023-11-02T16:15:17.536 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-02T16:15:17.537 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:15:17.550 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:15:17.539+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86408) UID: 0 2023-11-02T16:15:17.553 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-11-02T16:15:17.540+0000 b47e640 -1 osd.5 1042 *** Got signal Terminated *** 2023-11-02T16:15:19.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-11-02T16:15:23.639 INFO:tasks.ceph.osd.5:Stopped 2023-11-02T16:15:23.640 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-02T16:15:23.640 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:15:23.653 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:15:23.642+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86411) UID: 0 2023-11-02T16:15:23.653 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-11-02T16:15:23.644+0000 b47e640 -1 osd.6 1044 *** Got signal Terminated *** 2023-11-02T16:15:29.857 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:42.010 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.858 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:42.012 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.875 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:42.028 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.901 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:41.998 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.904 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:42.001 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.911 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:42.064 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.912 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:42.065 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.921 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:42.018 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:29.929 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:42.082 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.873 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:47.026 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.877 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:47.031 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.889 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:47.042 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.926 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:47.021 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.926 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:47.023 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.928 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:47.081 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.929 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:47.083 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.937 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:47.033 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:34.939 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:47.092 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.124 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:58.277 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.129 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:58.282 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.146 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:58.299 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.182 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:58.278 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.182 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:58.335 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.183 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:58.280 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.185 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:58.339 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.199 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:42:58.295 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:46.201 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:58.354 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.910 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:02.063 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.914 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:02.067 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.926 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:02.079 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.964 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:02.117 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.966 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:02.120 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.967 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:02.063 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.969 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:02.065 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.977 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:02.130 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:49.978 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:02.075 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.928 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:07.081 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.933 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:07.086 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.945 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:07.099 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.982 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:07.078 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.983 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:07.080 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.985 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:07.138 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.986 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:07.139 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.992 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:07.089 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:15:54.999 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:07.152 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:05.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-02T16:16:06.350 INFO:tasks.ceph.osd.6:Stopped 2023-11-02T16:16:06.351 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-02T16:16:06.351 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:06.364 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:16:06.353+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86413) UID: 0 2023-11-02T16:16:06.367 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-11-02T16:16:06.354+0000 b47e640 -1 osd.7 1044 *** Got signal Terminated *** 2023-11-02T16:16:09.970 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:22.123 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:09.973 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:22.127 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:09.995 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:22.148 85825== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.022 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:22.118 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.024 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:22.120 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.031 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:22.185 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.033 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:22.186 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.039 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:22.135 85872== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.048 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:22.202 85830== Warning: unimplemented fcntl command: 1036 2023-11-02T16:16:10.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-02T16:16:10.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-02T16:16:12.454 INFO:tasks.ceph.osd.7:Stopped 2023-11-02T16:16:12.454 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-02T16:16:12.455 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-02T16:16:12.455 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:12.456 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T16:16:12.454+0000 7f30d1094640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 85840) UID: 0 2023-11-02T16:16:12.456 INFO:tasks.ceph.mgr.y.smithi174.stderr:2023-11-02T16:16:12.454+0000 7f30d1094640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-02T16:16:12.482 INFO:tasks.ceph.mgr.y:Stopped 2023-11-02T16:16:12.483 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-02T16:16:12.483 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:12.508 INFO:tasks.ceph.mgr.x:Stopped 2023-11-02T16:16:12.509 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-02T16:16:12.509 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-02T16:16:12.509 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:12.541 INFO:tasks.ceph.mon.a.smithi174.stderr:2023-11-02T16:16:12.525+0000 9f34640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 85801) UID: 0 2023-11-02T16:16:12.542 INFO:tasks.ceph.mon.a.smithi174.stderr:2023-11-02T16:16:12.527+0000 9f34640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-02T16:16:13.166 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:25.319 85825== 2023-11-02T16:16:13.167 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:43:25.319 85825== Exit program on first error (--exit-on-first-error=yes) 2023-11-02T16:16:13.187 INFO:tasks.ceph.mon.a.smithi174.stderr:daemon-helper: command failed with exit status 42 2023-11-02T16:16:16.162 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-02T16:16:16.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-02T16:16:18.611 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-11-02T16:16:18.612 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi174 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-11-02T16:16:18.612 INFO:tasks.ceph.mon.a:Stopped 2023-11-02T16:16:18.613 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-02T16:16:18.613 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:18.629 INFO:tasks.ceph.mon.c.smithi174.stderr:2023-11-02T16:16:18.616+0000 9f34640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 85803) UID: 0 2023-11-02T16:16:18.636 INFO:tasks.ceph.mon.c.smithi174.stderr:2023-11-02T16:16:18.617+0000 9f34640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-11-02T16:16:19.238 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:31.391 85830== 2023-11-02T16:16:19.238 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:43:31.391 85830== Exit program on first error (--exit-on-first-error=yes) 2023-11-02T16:16:19.255 INFO:tasks.ceph.mon.c.smithi174.stderr:daemon-helper: command failed with exit status 42 2023-11-02T16:16:21.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-02T16:16:21.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-02T16:16:24.715 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-11-02T16:16:24.715 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi174 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-11-02T16:16:24.716 INFO:tasks.ceph.mon.c:Stopped 2023-11-02T16:16:24.717 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-02T16:16:24.717 INFO:teuthology.orchestra.run:waiting for 300 2023-11-02T16:16:24.733 INFO:tasks.ceph.mon.b.smithi175.stderr:2023-11-02T16:16:24.719+0000 9f34640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 85867) UID: 0 2023-11-02T16:16:24.733 INFO:tasks.ceph.mon.b.smithi175.stderr:2023-11-02T16:16:24.721+0000 9f34640 -1 mon.b@1(probing) e1 *** Got Signal Terminated *** 2023-11-02T16:16:25.273 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:37.370 85872== 2023-11-02T16:16:25.274 INFO:tasks.ceph.mon.b.smithi175.stderr:==00:00:43:37.370 85872== Exit program on first error (--exit-on-first-error=yes) 2023-11-02T16:16:25.293 INFO:tasks.ceph.mon.b.smithi175.stderr:daemon-helper: command failed with exit status 42 2023-11-02T16:16:26.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-02T16:16:26.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-02T16:16:30.819 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-11-02T16:16:30.819 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi175 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-11-02T16:16:30.820 INFO:tasks.ceph.mon.b:Stopped 2023-11-02T16:16:30.820 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-02T16:16:30.821 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-11-02T16:16:30.928 INFO:teuthology.orchestra.run.smithi174.stdout:2023-11-02T15:42:57.913880+0000 mon.a (mon.0) 1363 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2023-11-02T16:16:30.929 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-02T16:16:30.929 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-11-02T16:16:30.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-11-02T16:16:31.041 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-11-02T16:16:31.116 INFO:teuthology.orchestra.run.smithi174.stdout:2023-11-02T15:42:57.913880+0000 mon.a (mon.0) 1363 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2023-11-02T16:16:31.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T16:16:31.117 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-02T16:16:31.240 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T16:16:31.240 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-02T16:16:31.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-02T16:16:31.314 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T16:16:31.314 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-02T16:16:31.371 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T16:16:31.371 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-02T16:16:31.441 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T16:16:31.441 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-02T16:16:31.536 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T16:16:31.536 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-02T16:16:31.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T16:16:31.600 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-02T16:16:31.658 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T16:16:31.659 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-02T16:16:31.716 INFO:tasks.ceph:Archiving mon data... 2023-11-02T16:16:31.717 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/data/mon.a.tgz 2023-11-02T16:16:31.718 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2023-11-02T16:16:31.733 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.xbprreSfB9 2023-11-02T16:16:31.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xbprreSfB9 2023-11-02T16:16:31.929 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.xbprreSfB9 2023-11-02T16:16:32.014 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.xbprreSfB9 is 610KB 2023-11-02T16:16:32.116 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.xbprreSfB9 2023-11-02T16:16:32.131 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/data/mon.c.tgz 2023-11-02T16:16:32.132 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2023-11-02T16:16:32.184 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.5tBQbTFQcB 2023-11-02T16:16:32.184 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5tBQbTFQcB 2023-11-02T16:16:32.362 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.5tBQbTFQcB 2023-11-02T16:16:32.448 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.5tBQbTFQcB is 609KB 2023-11-02T16:16:32.553 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.5tBQbTFQcB 2023-11-02T16:16:32.569 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/data/mon.b.tgz 2023-11-02T16:16:32.569 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2023-11-02T16:16:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.6VIhRVlCsu 2023-11-02T16:16:32.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6VIhRVlCsu 2023-11-02T16:16:32.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.6VIhRVlCsu 2023-11-02T16:16:32.869 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.6VIhRVlCsu is 609KB 2023-11-02T16:16:32.972 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.6VIhRVlCsu 2023-11-02T16:16:32.987 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-02T16:16:32.987 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-02T16:16:32.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-02T16:16:33.052 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-02T16:16:33.053 DEBUG:teuthology.orchestra.run.smithi174:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-02T16:16:33.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-02T16:16:33.096 INFO:teuthology.orchestra.run.smithi174.stdout:/var/log/ceph/valgrind/mon.a.log: Leak_StillReachable 2023-11-02T16:16:33.097 INFO:teuthology.orchestra.run.smithi174.stdout:/var/log/ceph/valgrind/mon.c.log: Leak_StillReachable 2023-11-02T16:16:33.097 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.a.log kind Leak_StillReachable 2023-11-02T16:16:33.098 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.a.log 2023-11-02T16:16:33.098 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.c.log kind Leak_StillReachable 2023-11-02T16:16:33.098 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.c.log 2023-11-02T16:16:33.132 INFO:teuthology.orchestra.run.smithi175.stdout:/var/log/ceph/valgrind/mon.b.log: Leak_StillReachable 2023-11-02T16:16:33.132 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.b.log kind Leak_StillReachable 2023-11-02T16:16:33.132 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.b.log 2023-11-02T16:16:33.133 INFO:tasks.ceph:Archiving crash dumps... 2023-11-02T16:16:33.134 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi174/crash 2023-11-02T16:16:33.136 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-02T16:16:33.164 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi175/crash 2023-11-02T16:16:33.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-02T16:16:33.198 INFO:tasks.ceph:Compressing logs... 2023-11-02T16:16:33.198 DEBUG:teuthology.orchestra.run.smithi174:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-02T16:16:33.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-02T16:16:58.350 INFO:tasks.ceph:Archiving logs... 2023-11-02T16:16:58.352 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi174/log 2023-11-02T16:16:58.353 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-02T16:17:01.752 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi175/log 2023-11-02T16:17:01.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-02T16:17:04.297 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-11-02T16:17:04.298 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-02T16:17:04.308 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 1951, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b5a9486b4a6f144f83c51b7ec33111a2c86dc081/qa/tasks/ceph.py", line 362, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-11-02T16:17:04.308 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-02T16:17:04.309 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-02T16:17:04.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-02T16:17:04.397 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-02T16:17:04.409 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-02T16:17:04.410 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-02T16:17:04.414 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-02T16:17:04.428 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2023-11-02T16:17:04.430 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2023-11-02T16:17:04.433 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-02T16:17:04.434 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-02T16:17:04.434 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +1948us[+1932us] +/- 73ms 2023-11-02T16:17:04.434 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -659us[ -675us] +/- 55ms 2023-11-02T16:17:04.434 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -874us[ -874us] +/- 64ms 2023-11-02T16:17:04.435 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T16:17:04.438 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-02T16:17:04.438 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-11-02T16:17:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +1966us[+1966us] +/- 73ms 2023-11-02T16:17:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -649us[ -640us] +/- 55ms 2023-11-02T16:17:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -869us[ -869us] +/- 64ms 2023-11-02T16:17:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-02T16:17:04.440 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-02T16:17:04.450 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-02T16:17:04.452 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-02T16:17:04.464 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-02T16:17:04.578 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-02T16:17:04.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-02T16:17:04.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T16:17:04.709 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-02T16:17:04.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T16:17:04.742 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-02T16:17:04.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-02T16:17:04.766 INFO:teuthology.task.internal:Duration was 3623.719235 seconds 2023-11-02T16:17:04.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-02T16:17:04.782 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-02T16:17:04.783 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-02T16:17:04.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-02T16:17:04.827 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-02T16:17:04.830 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-02T16:17:05.141 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-02T16:17:05.142 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2023-11-02T16:17:05.142 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-02T16:17:05.201 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2023-11-02T16:17:05.202 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-02T16:17:05.232 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-02T16:17:05.232 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-02T16:17:05.244 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-02T16:17:05.302 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-02T16:17:05.303 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-02T16:17:05.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-02T16:17:05.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-02T16:17:05.449 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-02T16:17:05.450 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-02T16:17:05.477 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-02T16:17:05.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-02T16:17:05.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-02T16:17:05.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-02T16:17:05.544 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2023-11-02T16:17:05.572 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2023-11-02T16:17:05.597 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-02T16:17:05.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T16:17:05.616 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-02T16:17:05.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-02T16:17:05.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-02T16:17:05.662 INFO:teuthology.task.internal:Transferring archived files... 2023-11-02T16:17:05.664 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi174 2023-11-02T16:17:05.664 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-02T16:17:05.701 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599/remote/smithi175 2023-11-02T16:17:05.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-02T16:17:05.738 INFO:teuthology.task.internal:Removing archive directory... 2023-11-02T16:17:05.739 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-02T16:17:05.741 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-02T16:17:05.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-02T16:17:05.802 INFO:teuthology.task.internal:Not uploading archives. 2023-11-02T16:17:05.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-02T16:17:05.819 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-02T16:17:05.820 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-02T16:17:05.824 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-02T16:17:05.835 INFO:teuthology.orchestra.run.smithi174.stdout: 265772 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 2 16:17 /home/ubuntu/cephtest 2023-11-02T16:17:05.841 INFO:teuthology.orchestra.run.smithi175.stdout: 265781 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 2 16:17 /home/ubuntu/cephtest 2023-11-02T16:17:05.843 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-02T16:17:05.855 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-02T16:17:05.876 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: saw valgrind issues 2023-11-02T16:17:05.876 INFO:teuthology.nuke:Checking targets against current locks 2023-11-02T16:17:05.905 DEBUG:teuthology.nuke:shortname: smithi174 2023-11-02T16:17:05.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-02T16:17:05.927 DEBUG:teuthology.nuke:shortname: smithi175 2023-11-02T16:17:05.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-02T16:17:05.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-02 15:08:37.119118', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBugmfG+rwui8Ts34mPlTCz0WfNTdtwY57m8tJjMFcAuonN4c0UAtSWyRtuAfmNqAPq1XJ7wLMHJ0GnsykGZZe8='} 2023-11-02T16:17:05.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-02_14:20:05-rados-wip-yuri6-testing-2023-11-01-0745-reef-distro-default-smithi/7444599', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-02 15:08:37.120312', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgiUDqKH+Rx2Vp7SONDWsX0Ye5bRtCPgwzqtDCC/9kP7WEpPClEKou7KBljVcDfFamD9gd/+RNBQ/+LSXdMf10='} 2023-11-02T16:17:05.982 INFO:teuthology.orchestra.console.smithi174:Power off 2023-11-02T16:17:05.983 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-02T16:17:06.009 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2023-11-02T16:17:06.009 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:06.026 INFO:teuthology.orchestra.console.smithi175:Power off 2023-11-02T16:17:06.026 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-02T16:17:06.038 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-02T16:17:06.050 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2023-11-02T16:17:06.051 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:06.076 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-11-02T16:17:10.039 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:10.078 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:10.149 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-02T16:17:10.194 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-11-02T16:17:14.154 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:14.195 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:14.264 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-02T16:17:14.306 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-11-02T16:17:18.267 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:18.307 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-02T16:17:18.382 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-11-02T16:17:18.420 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2023-11-02T16:17:18.484 INFO:teuthology.orchestra.console.smithi174:Power off completed 2023-11-02T16:17:18.521 INFO:teuthology.orchestra.console.smithi175:Power off completed 2023-11-02T16:17:18.626 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} duration: 3623.719235420227 failure_reason: '"2023-11-02T15:42:57.913880+0000 mon.a (mon.0) 1363 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-11-02T16:17:18.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-02T16:17:18.716 INFO:teuthology.run:FAIL